Skip to contentSkip to conversation

Add rust runtime tests

PATCH Superseded Alex Kiernan 2 participants 18 messages Dec 25, 2022 – Dec 29, 2022

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Add rust runtime tests

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

 meta/lib/oeqa/runtime/cases/rust.py           |  17 ++
 .../packagegroups/packagegroup-core-sdk.bb    |  10 +-
 .../packagegroup-rust-sdk-target.bb           |  14 ++
 meta/recipes-devtools/cargo/cargo.inc         |  69 ------
 meta/recipes-devtools/cargo/cargo_1.66.0.bb   |  80 ++++++-
 meta/recipes-devtools/rust/rust-llvm.inc      |  78 ------
 .../recipes-devtools/rust/rust-llvm_1.66.0.bb |  82 ++++++-
 meta/recipes-devtools/rust/rust-source.inc    |   5 +
 meta/recipes-devtools/rust/rust-target.inc    |  10 -
 meta/recipes-devtools/rust/rust.inc           | 223 ------------------
 meta/recipes-devtools/rust/rust_1.66.0.bb     | 223 +++++++++++++++++-
 11 files changed, 420 insertions(+), 391 deletions(-)
 create mode 100644 meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb
 delete mode 100644 meta/recipes-devtools/cargo/cargo.inc
 delete mode 100644 meta/recipes-devtools/rust/rust-llvm.inc
 delete mode 100644 meta/recipes-devtools/rust/rust-target.inc
 delete mode 100644 meta/recipes-devtools/rust/rust.inc
Signature · 1 line
2.39.0
#1 lore raw

PATCH v4 1/11 rust-llvm: Merge .inc into .bb

---

(no changes since v2)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

 meta/recipes-devtools/rust/rust-llvm.inc      | 78 ------------------
 .../recipes-devtools/rust/rust-llvm_1.66.0.bb | 80 ++++++++++++++++++-
 2 files changed, 79 insertions(+), 79 deletions(-)
 delete mode 100644 meta/recipes-devtools/rust/rust-llvm.inc
meta/recipes-devtools/rust/rust-llvm.inc deleted +0 -78
deleted file mode 100644
index 625eb5704166..000000000000
@@ -1,78 +0,0 @@
-SUMMARY = "LLVM compiler framework (packaged with rust)"
-LICENSE ?= "Apache-2.0-with-LLVM-exception"
-HOMEPAGE = "http://www.rust-lang.org"
-
-SRC_URI += "file://0002-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \
-            file://0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch;striplevel=2 \
-	    file://0003-llvm-fix-include-benchmarks.patch;striplevel=2"
-
-S = "${RUSTSRC}/src/llvm-project/llvm"
-
-LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=8a15a0759ef07f2682d2ba4b893c9afe"
-
-inherit cmake python3native
-
-DEPENDS += "ninja-native rust-llvm-native"
-
-ARM_INSTRUCTION_SET:armv5 = "arm"
-ARM_INSTRUCTION_SET:armv4t = "arm"
-
-# rustc_llvm with debug info is not recognized as a valid crate that's
-# generated by rust-llvm-native.
-CFLAGS:remove = "-g"
-CXXFLAGS:remove = "-g"
-
-LLVM_DIR = "llvm${LLVM_RELEASE}"
-
-EXTRA_OECMAKE = " \
-    -DCMAKE_BUILD_TYPE=Release \
-    -DLLVM_TARGETS_TO_BUILD='ARM;AArch64;Mips;PowerPC;RISCV;X86' \
-    -DLLVM_BUILD_DOCS=OFF \
-    -DLLVM_ENABLE_TERMINFO=OFF \
-    -DLLVM_ENABLE_ZLIB=OFF \
-    -DLLVM_ENABLE_LIBXML2=OFF \
-    -DLLVM_ENABLE_FFI=OFF \
-    -DLLVM_INSTALL_UTILS=ON \
-    -DLLVM_BUILD_EXAMPLES=OFF \
-    -DLLVM_INCLUDE_EXAMPLES=OFF \
-    -DLLVM_BUILD_TESTS=OFF \
-    -DLLVM_INCLUDE_TESTS=OFF \
-    -DLLVM_TARGET_ARCH=${TARGET_ARCH} \
-    -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust \
-"
-EXTRA_OECMAKE:append:class-target = "\
-    -DCMAKE_CROSSCOMPILING:BOOL=ON \
-    -DLLVM_BUILD_TOOLS=OFF \
-    -DLLVM_TABLEGEN=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-tblgen \
-    -DLLVM_CONFIG_PATH=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config \
-"
-
-EXTRA_OECMAKE:append:class-nativesdk = "\
-    -DCMAKE_CROSSCOMPILING:BOOL=ON \
-    -DLLVM_BUILD_TOOLS=OFF \
-    -DLLVM_TABLEGEN=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-tblgen \
-    -DLLVM_CONFIG_PATH=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config \
-"
-
-# The debug symbols are huge here (>2GB) so suppress them since they
-# provide almost no value. If you really need them then override this
-INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
-
-export YOCTO_ALTERNATE_EXE_PATH = "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
-
-do_install:append () {
-    # we don't need any of this stuff to build Rust
-    rm -rf "${D}/usr/lib/cmake"
-}
-
-PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liblto"
-
-# Add the extra locations to avoid the complaints about unpackaged files
-FILES:${PN}-bugpointpasses = "${libdir}/llvm-rust/lib/BugpointPasses.so"
-FILES:${PN}-llvmhello = "${libdir}/llvm-rust/lib/LLVMHello.so"
-FILES:${PN}-liblto = "${libdir}/llvm-rust/lib/libLTO.so.*"
-FILES:${PN}-staticdev =+ "${libdir}/llvm-rust/*/*.a"
-FILES:${PN} += "${libdir}/libLLVM*.so.* ${libdir}/llvm-rust/lib/*.so.* ${libdir}/llvm-rust/bin"
-FILES:${PN}-dev += "${datadir}/llvm ${libdir}/llvm-rust/lib/*.so ${libdir}/llvm-rust/include ${libdir}/llvm-rust/share ${libdir}/llvm-rust/lib/cmake"
-
-BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/rust/rust-llvm_1.66.0.bb +79 -1

85 lines not on this page · raw message

Signature · 1 line
2.39.0
#2 lore raw

PATCH v4 2/11 rust-llvm: Update LLVM_VERSION to match embedded version

In reply to Alex Kiernan

---

(no changes since v1)

 meta/recipes-devtools/rust/rust-llvm_1.66.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/rust/rust-llvm_1.66.0.bb +1 -1
index e1baeb78574f..4cf244bb67f5 100644
@@ -4,7 +4,7 @@ HOMEPAGE = "http://www.rust-lang.org"
 
 # check src/llvm-project/llvm/CMakeLists.txt for llvm version in use
 #
-LLVM_RELEASE = "14.0.5"
+LLVM_RELEASE = "15.0.2"
 
 require rust-source.inc
 
Signature · 1 line
2.39.0
#3 lore raw

PATCH v4 3/11 packagegroup-rust-sdk-target: Add Rust SDK target packagegroup

In reply to Alex Kiernan

---

(no changes since v1)

 .../packagegroups/packagegroup-rust-sdk-target.bb   | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100644 meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb
meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb new file +13 -0
new file mode 100644
index 000000000000..334b24900b4b
@@ -0,0 +1,13 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+SUMMARY = "Target packages for the Rust SDK"
+
+inherit packagegroup
+
+RDEPENDS:${PN} = " \
+    rust \
+"
Signature · 1 line
2.39.0
#4 lore raw

PATCH v4 4/11 packagegroup-core-sdk: Add SDK toolchain language selection support

In reply to Alex Kiernan

Use SDK_TOOLCHAIN_LANGS to allow different language support to be selected within SDKs. Initially supported options are rust and go.

---

(no changes since v3)

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

 .../packagegroups/packagegroup-core-sdk.bb             | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
meta/recipes-core/packagegroups/packagegroup-core-sdk.bb +9 -1
index d70aff22c725..104f3549d593 100644
@@ -11,6 +11,11 @@ inherit packagegroup
 
 #PACKAGEFUNCS =+ 'generate_sdk_pkgs'
 
+TARGET_TOOLCHAIN_LANGS ??= "${SDK_TOOLCHAIN_LANGS}"
+TARGET_TOOLCHAIN_LANGS:remove:sdkmingw32 = "rust"
+# libstd-rs doesn't build for mips n32 with compiler constraint errors
+TARGET_TOOLCHAIN_LANGS:remove:mipsarchn32 = "rust"
+
 RDEPENDS:packagegroup-core-sdk = "\
     packagegroup-core-buildessential \
     coreutils \
@@ -23,7 +28,10 @@ RDEPENDS:packagegroup-core-sdk = "\
     less \
     ldd \
     file \
-    tcl"
+    tcl \
+    ${@bb.utils.contains('TARGET_TOOLCHAIN_LANGS', 'go', 'packagegroup-go-sdk-target', '', d)} \
+    ${@bb.utils.contains('TARGET_TOOLCHAIN_LANGS', 'rust', 'packagegroup-rust-sdk-target', '', d)} \
+"
 
 SANITIZERS = "libasan-dev libubsan-dev"
 SANITIZERS:arc = ""
Signature · 1 line
2.39.0
#5 lore raw

PATCH v4 5/11 cargo: Merge .inc into .bb

In reply to Alex Kiernan

---

(no changes since v1)

 meta/recipes-devtools/cargo/cargo.inc       | 69 --------------------
 meta/recipes-devtools/cargo/cargo_1.66.0.bb | 71 ++++++++++++++++++++-
 2 files changed, 69 insertions(+), 71 deletions(-)
 delete mode 100644 meta/recipes-devtools/cargo/cargo.inc
meta/recipes-devtools/cargo/cargo.inc deleted +0 -69
deleted file mode 100644
index 40421df4f76d..000000000000
@@ -1,69 +0,0 @@
-SUMMARY ?= "Cargo, a package manager for Rust."
-HOMEPAGE = "https://crates.io"
-LICENSE = "MIT | Apache-2.0"
-SECTION = "devel"
-
-DEPENDS = "openssl zlib curl ca-certificates libssh2"
-
-LIC_FILES_CHKSUM = " \
-    file://LICENSE-MIT;md5=b377b220f43d747efdec40d69fcaa69d \
-    file://LICENSE-APACHE;md5=71b224ca933f0676e26d5c2e2271331c \
-    file://LICENSE-THIRD-PARTY;md5=f257ad009884cb88a3a87d6920e7180a \
-"
-
-
-S = "${RUSTSRC}/src/tools/cargo"
-CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
-EXCLUDE_FROM_WORLD = "1"
-
-inherit cargo pkgconfig
-
-do_cargo_setup_snapshot () {
-	${WORKDIR}/rust-snapshot-components/${CARGO_SNAPSHOT}/install.sh --prefix="${WORKDIR}/${CARGO_SNAPSHOT}" --disable-ldconfig
-	# Need to use uninative's loader if enabled/present since the library paths
-	# are used internally by rust and result in symbol mismatches if we don't
-	if [ ! -z "${UNINATIVE_LOADER}" -a -e "${UNINATIVE_LOADER}" ]; then
-		patchelf-uninative ${WORKDIR}/${CARGO_SNAPSHOT}/bin/cargo --set-interpreter ${UNINATIVE_LOADER}
-	fi
-}
-
-addtask cargo_setup_snapshot after do_unpack before do_configure
-do_cargo_setup_snapshot[dirs] += "${WORKDIR}/${CARGO_SNAPSHOT}"
-do_cargo_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER"
-
-
-do_compile:prepend () {
-	export RUSTC_BOOTSTRAP="1"
-}
-
-do_install () {
-	install -d "${D}${bindir}"
-	install -m 755 "${B}/target/${CARGO_TARGET_SUBDIR}/cargo" "${D}${bindir}"
-}
-
-do_install:append:class-nativesdk() {
-	# To quote the cargo docs, "Cargo also sets the dynamic library path when compiling
-	# and running binaries with commands like `cargo run` and `cargo test`". Sadly it
-	# sets to libdir but not base_libdir leading to symbol mismatches depending on the
-	# host OS. Fully set LD_LIBRARY_PATH to contain both to avoid this.
-	create_wrapper ${D}/${bindir}/cargo LD_LIBRARY_PATH=${libdir}:${base_libdir}
-}
-
-# Disabled due to incompatibility with libgit2 0.28.x (https://github.com/rust-lang/git2-rs/issues/458, https://bugs.gentoo.org/707746#c1)
-# as shipped by Yocto Dunfell.
-# According to https://github.com/rust-lang/git2-rs/issues/458#issuecomment-522567539, there are no compatibility guarantees between
-# libgit2-sys and arbitrary system libgit2 versions, so better keep this turned off.
-#export LIBGIT2_SYS_USE_PKG_CONFIG = "1"
-
-# Needed for pkg-config to be used
-export LIBSSH2_SYS_USE_PKG_CONFIG = "1"
-
-# When building cargo-native we don't have cargo-native to use and depend on,
-# so we must use the locally set up snapshot to bootstrap the build.
-BASEDEPENDS:remove:class-native = "cargo-native"
-CARGO:class-native = "${WORKDIR}/${CARGO_SNAPSHOT}/bin/cargo"
-
-DEPENDS:append:class-nativesdk = " nativesdk-rust"
-RUSTLIB:append:class-nativesdk = " -L ${STAGING_DIR_HOST}/${SDKPATHNATIVE}/usr/lib/rustlib/${RUST_HOST_SYS}/lib"
-
-
meta/recipes-devtools/cargo/cargo_1.66.0.bb +69 -2

75 lines not on this page · raw message

Signature · 1 line
2.39.0
#6 lore raw

PATCH v4 6/11 cargo: Extend DEBUG_PREFIX_MAP to cover vendor

In reply to Alex Kiernan

The cargo build builds vendored libgit and curl, but these exist outside ${S} which DEBUG_PREFIX_MAP covers.

---

(no changes since v1)

 meta/recipes-devtools/cargo/cargo_1.66.0.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/cargo/cargo_1.66.0.bb +2 -0
index 6c167c0c5f0b..fe8049b68c9b 100644
@@ -20,6 +20,8 @@ EXCLUDE_FROM_WORLD = "1"
 
 inherit cargo pkgconfig
 
+DEBUG_PREFIX_MAP += "-fdebug-prefix-map=${RUSTSRC}/vendor=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}"
+
 do_cargo_setup_snapshot () {
 	${WORKDIR}/rust-snapshot-components/${CARGO_SNAPSHOT}/install.sh --prefix="${WORKDIR}/${CARGO_SNAPSHOT}" --disable-ldconfig
 	# Need to use uninative's loader if enabled/present since the library paths
Signature · 1 line
2.39.0
#7 lore raw

PATCH v4 7/11 rust: Merge .inc into .bb

In reply to Alex Kiernan

---

(no changes since v1)

 meta/recipes-devtools/rust/rust-target.inc |  10 -
 meta/recipes-devtools/rust/rust.inc        | 223 ---------------------
 meta/recipes-devtools/rust/rust_1.66.0.bb  | 218 +++++++++++++++++++-
 3 files changed, 217 insertions(+), 234 deletions(-)
 delete mode 100644 meta/recipes-devtools/rust/rust-target.inc
 delete mode 100644 meta/recipes-devtools/rust/rust.inc
meta/recipes-devtools/rust/rust-target.inc deleted +0 -10
deleted file mode 100644
index dce2b47517b5..000000000000
@@ -1,10 +0,0 @@
-require rust.inc
-
-DEPENDS += "rust-llvm (=${PV})"
-
-# Otherwise we'll depend on what we provide
-INHIBIT_DEFAULT_RUST_DEPS:class-native = "1"
-# We don't need to depend on gcc-native because yocto assumes it exists
-PROVIDES:class-native = "virtual/${TARGET_PREFIX}rust"
-
-BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/rust/rust.inc deleted +0 -223

223 lines not on this page · raw message

meta/recipes-devtools/rust/rust_1.66.0.bb +217 -1

230 lines not on this page · raw message

Signature · 1 line
2.39.0
#8 lore raw

PATCH v4 8/11 rust: Move musl-x86 fix for `__stack_chk_fail_local` to rust-source

In reply to Alex Kiernan

Any consumer of rust-source (potentially) needs this, so move to rust-source.inc

---

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86

 meta/recipes-devtools/rust/rust-source.inc | 5 +++++
 meta/recipes-devtools/rust/rust_1.66.0.bb  | 5 -----
 2 files changed, 5 insertions(+), 5 deletions(-)
meta/recipes-devtools/rust/rust-source.inc +5 -0
index 91a6c5a942ae..b63e18ac95d2 100644
@@ -11,3 +11,8 @@ RUSTSRC = "${WORKDIR}/rustc-${RUST_VERSION}-src"
 
 UPSTREAM_CHECK_URI = "https://forge.rust-lang.org/infra/other-installation-methods.html"
 UPSTREAM_CHECK_REGEX = "rustc-(?P<pver>\d+(\.\d+)+)-src"
+
+# see recipes-devtools/gcc/gcc/0018-Add-ssp_nonshared-to-link-commandline-for-musl-targe.patch
+# we need to link with ssp_nonshared on musl to avoid "undefined reference to `__stack_chk_fail_local'"
+# when building MACHINE=qemux86 for musl
+WRAPPER_TARGET_EXTRALD:libc-musl = "-lssp_nonshared"
meta/recipes-devtools/rust/rust_1.66.0.bb +0 -5
index 5192ec2ee11a..f1ec769aef48 100644
@@ -292,11 +292,6 @@ rust_do_install:class-target() {
     rm ${D}${libdir}/rustlib/manifest*
 }
 
-# see recipes-devtools/gcc/gcc/0018-Add-ssp_nonshared-to-link-commandline-for-musl-targe.patch
-# we need to link with ssp_nonshared on musl to avoid "undefined reference to `__stack_chk_fail_local'"
-# when building MACHINE=qemux86 for musl
-WRAPPER_TARGET_EXTRALD:libc-musl = "-lssp_nonshared"
-
 RUSTLIB_DEP:class-nativesdk = ""
 
 # musl builds include libunwind.a
Signature · 1 line
2.39.0
#9 lore raw

PATCH v4 9/11 cargo: Override crossbeam-utils with 0.8.14

In reply to Alex Kiernan

Use cargo patching to replace crossbeam-utils with the semver compatible 0.8.14 rather than trying to patch our local changes. This corrects the ppc/mips builds.

---

Changes in v4:
- Add crossbeam atomics fixes for ppc/mips

 meta/recipes-devtools/cargo/cargo_1.66.0.bb | 7 +++++++
 1 file changed, 7 insertions(+)
meta/recipes-devtools/cargo/cargo_1.66.0.bb +7 -0
index fe8049b68c9b..909651b9efdd 100644
@@ -14,6 +14,13 @@ LIC_FILES_CHKSUM = " \
 require recipes-devtools/rust/rust-source.inc
 require recipes-devtools/rust/rust-snapshot.inc
 
+# Reuse the path override capability we have to patch in a later version
+# of crossbeam-utils which has the vendor misdetection fixes
+# (https://github.com/crossbeam-rs/crossbeam/pull/922). This should be
+# removed once the upstream source merges the change to include it.
+SRC_URI:append:class-target = " crate://crates.io/crossbeam-utils/0.8.14"
+EXTRA_OECARGO_PATHS:class-target = "${CARGO_HOME}/bitbake/crossbeam-utils-0.8.14"
+
 S = "${RUSTSRC}/src/tools/cargo"
 CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
 EXCLUDE_FROM_WORLD = "1"
Signature · 1 line
2.39.0
#10 lore raw

PATCH v4 10/11 packagegroup-rust-sdk-target: Add cargo

In reply to Alex Kiernan

---

(no changes since v1)

 meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb +1 -0
index 334b24900b4b..59874c4c2c81 100644
@@ -10,4 +10,5 @@ inherit packagegroup
 
 RDEPENDS:${PN} = " \
     rust \
+    cargo \
 "
Signature · 1 line
2.39.0
#11 lore raw

PATCH v4 11/11 oeqa/runtime/rust: Add cargo test

In reply to Alex Kiernan

---

(no changes since v1)

 meta/lib/oeqa/runtime/cases/rust.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
meta/lib/oeqa/runtime/cases/rust.py +17 -0
index 186bb0d79e15..c9c60e16fd2b 100644
@@ -20,6 +20,8 @@ class RustCompileTest(OERuntimeTestCase):
     def tearDown(cls):
         files = '/tmp/test.rs /tmp/test'
         cls.tc.target.run('rm %s' % files)
+        dirs = '/tmp/hello'
+        cls.tc.target.run('rm -r %s' % dirs)
 
     @OETestDepends(['ssh.SSHTest.test_ssh'])
     @OEHasPackage(['rust'])
@@ -32,6 +34,21 @@ class RustCompileTest(OERuntimeTestCase):
         msg = 'running compiled file failed, output: %s' % output
         self.assertEqual(status, 0, msg=msg)
 
+    @OETestDepends(['ssh.SSHTest.test_ssh'])
+    @OEHasPackage(['cargo'])
+    def test_cargo_compile(self):
+        status, output = self.target.run('cargo new /tmp/hello')
+        msg = 'cargo new failed, output: %s' % output
+        self.assertEqual(status, 0, msg=msg)
+
+        status, output = self.target.run('cargo build --manifest-path=/tmp/hello/Cargo.toml')
+        msg = 'cargo build failed, output: %s' % output
+        self.assertEqual(status, 0, msg=msg)
+
+        status, output = self.target.run('cargo run --manifest-path=/tmp/hello/Cargo.toml')
+        msg = 'running compiled file failed, output: %s' % output
+        self.assertEqual(status, 0, msg=msg)
+
 class RustHelloworldTest(OERuntimeTestCase):
     @OETestDepends(['ssh.SSHTest.test_ssh'])
     @OEHasPackage(['rust-hello-world'])
Signature · 1 line
2.39.0
#12 lore raw
In reply to Alex Kiernan
Quoted 19 lines · Alex Kiernan

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Quoted 12 lines

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio

We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.

Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.

Cc Sundeep who as been looking at it.

Cheers,

Richard

#13 lore raw
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Quoted 12 lines

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio

We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.

Quoted 6 lines

Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.

Cc Sundeep who as been looking at it.

Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.

Can I get hold of the cargo packages generated here to have a poke at them?

Signature · 1 line
Alex Kiernan
#14 lore raw
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Quoted 12 lines

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio

We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.

Quoted 6 lines

Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.

Cc Sundeep who as been looking at it.
Quoted 8 lines

Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.

Can I get hold of the cargo packages generated here to have a poke at them?

Yes. The key line in the log is:

022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to /srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

i.e.

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

where you'll find both the package files and diffoscope output:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/

which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.

Cheers,

Richard

#15 lore raw
Quoted 70 lines · Richard Purdie
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Quoted 12 lines

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio

We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.

Quoted 6 lines

Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.

Cc Sundeep who as been looking at it.
Quoted 8 lines

Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.

Can I get hold of the cargo packages generated here to have a poke at them?

Yes. The key line in the log is:

022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to /srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

i.e.

Quoted 9 lines

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

where you'll find both the package files and diffoscope output:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/

which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.

Upstream has this ticket which, given how I did the patching for cargo, feels like the right kind of thing (rust has something similar out of the box):

https://github.com/rust-lang/rust/issues/98185

I've switched it around so that it uses local patching rather than cargo patching, which if it is this, seems like it might resolve it.

--
Alex Kiernan

#16 lore raw
Quoted 82 lines · Alex Kiernan
Quoted 70 lines · Richard Purdie
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Quoted 12 lines

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio

We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.

Quoted 6 lines

Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.

Cc Sundeep who as been looking at it.
Quoted 8 lines

Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.

Can I get hold of the cargo packages generated here to have a poke at them?

Yes. The key line in the log is:

022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to /srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

i.e.

Quoted 9 lines

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

where you'll find both the package files and diffoscope output:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/

which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.

Quoted 8 lines

Upstream has this ticket which, given how I did the patching for cargo, feels like the right kind of thing (rust has something similar out of the box):

https://github.com/rust-lang/rust/issues/98185

I've switched it around so that it uses local patching rather than cargo patching, which if it is this, seems like it might resolve it.

It was a different issue, thanks for tracking that down and fixing it!

The tests all passed so I merged the v5 series. Thanks for working through things, I think this makes our rust situation much more robust and should help future proof it :)

Cheers,

Richard

#17 lore raw
Quoted 88 lines · Richard Purdie
Quoted 86 lines · Alex Kiernan
Quoted 70 lines · Richard Purdie
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan

This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.

Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.

Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips

Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)

Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes

Quoted 12 lines

Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test

Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio

We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.

Quoted 6 lines

Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.

Cc Sundeep who as been looking at it.
Quoted 8 lines

Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.

Can I get hold of the cargo packages generated here to have a poke at them?

Yes. The key line in the log is:

022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to /srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

i.e.

Quoted 9 lines

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk

where you'll find both the package files and diffoscope output:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/

which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.

Upstream has this ticket which, given how I did the patching for cargo, feels like the right kind of thing (rust has something similar out of the box):

Quoted 7 lines
Quoted 4 lines

https://github.com/rust-lang/rust/issues/98185

I've switched it around so that it uses local patching rather than cargo patching, which if it is this, seems like it might resolve it.

It was a different issue, thanks for tracking that down and fixing it!

I'm not so sure - there's a .cargo/config.toml in the vendored rustc tarball which explicitly patches one of the vendored sources (it feels odd, but hey...), so I think we'll trip over this issue (assuming that's what the problem is).

Quoted 4 lines

The tests all passed so I merged the v5 series. Thanks for working through things, I think this makes our rust situation much more robust and should help future proof it :)

:)

Signature · 1 line
Alex Kiernan