Add rust runtime tests
v4 cover Add rust runtime tests
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add SDK_TOOLCHAIN_LANGS for selection of target toolchains as part of packagegroup-core-sdk.
Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
- Add crossbeam atomics fixes for ppc/mips
Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)
Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: Update LLVM_VERSION to match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: Extend DEBUG_PREFIX_MAP to cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for __stack_chk_fail_local to rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test
meta/lib/oeqa/runtime/cases/rust.py | 17 ++ .../packagegroups/packagegroup-core-sdk.bb | 10 +- .../packagegroup-rust-sdk-target.bb | 14 ++ meta/recipes-devtools/cargo/cargo.inc | 69 ------ meta/recipes-devtools/cargo/cargo_1.66.0.bb | 80 ++++++- meta/recipes-devtools/rust/rust-llvm.inc | 78 ------ .../recipes-devtools/rust/rust-llvm_1.66.0.bb | 82 ++++++- meta/recipes-devtools/rust/rust-source.inc | 5 + meta/recipes-devtools/rust/rust-target.inc | 10 - meta/recipes-devtools/rust/rust.inc | 223 ------------------ meta/recipes-devtools/rust/rust_1.66.0.bb | 223 +++++++++++++++++- 11 files changed, 420 insertions(+), 391 deletions(-) create mode 100644 meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb delete mode 100644 meta/recipes-devtools/cargo/cargo.inc delete mode 100644 meta/recipes-devtools/rust/rust-llvm.inc delete mode 100644 meta/recipes-devtools/rust/rust-target.inc delete mode 100644 meta/recipes-devtools/rust/rust.inc
Signature · 1 line
2.39.0
PATCH v4 1/11 rust-llvm: Merge .inc into .bb
---
(no changes since v2)
Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes
meta/recipes-devtools/rust/rust-llvm.inc | 78 ------------------ .../recipes-devtools/rust/rust-llvm_1.66.0.bb | 80 ++++++++++++++++++- 2 files changed, 79 insertions(+), 79 deletions(-) delete mode 100644 meta/recipes-devtools/rust/rust-llvm.inc
meta/recipes-devtools/rust/rust-llvm.inc deleted +0 -78
@@ -1,78 +0,0 @@ -SUMMARY = "LLVM compiler framework (packaged with rust)" -LICENSE ?= "Apache-2.0-with-LLVM-exception" -HOMEPAGE = "http://www.rust-lang.org" - -SRC_URI += "file://0002-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \ - file://0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch;striplevel=2 \ - file://0003-llvm-fix-include-benchmarks.patch;striplevel=2" - -S = "${RUSTSRC}/src/llvm-project/llvm" - -LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=8a15a0759ef07f2682d2ba4b893c9afe" - -inherit cmake python3native - -DEPENDS += "ninja-native rust-llvm-native" - -ARM_INSTRUCTION_SET:armv5 = "arm" -ARM_INSTRUCTION_SET:armv4t = "arm" - -# rustc_llvm with debug info is not recognized as a valid crate that's -# generated by rust-llvm-native. -CFLAGS:remove = "-g" -CXXFLAGS:remove = "-g" - -LLVM_DIR = "llvm${LLVM_RELEASE}" - -EXTRA_OECMAKE = " \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_TARGETS_TO_BUILD='ARM;AArch64;Mips;PowerPC;RISCV;X86' \ - -DLLVM_BUILD_DOCS=OFF \ - -DLLVM_ENABLE_TERMINFO=OFF \ - -DLLVM_ENABLE_ZLIB=OFF \ - -DLLVM_ENABLE_LIBXML2=OFF \ - -DLLVM_ENABLE_FFI=OFF \ - -DLLVM_INSTALL_UTILS=ON \ - -DLLVM_BUILD_EXAMPLES=OFF \ - -DLLVM_INCLUDE_EXAMPLES=OFF \ - -DLLVM_BUILD_TESTS=OFF \ - -DLLVM_INCLUDE_TESTS=OFF \ - -DLLVM_TARGET_ARCH=${TARGET_ARCH} \ - -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust \ -" -EXTRA_OECMAKE:append:class-target = "\ - -DCMAKE_CROSSCOMPILING:BOOL=ON \ - -DLLVM_BUILD_TOOLS=OFF \ - -DLLVM_TABLEGEN=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-tblgen \ - -DLLVM_CONFIG_PATH=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config \ -" - -EXTRA_OECMAKE:append:class-nativesdk = "\ - -DCMAKE_CROSSCOMPILING:BOOL=ON \ - -DLLVM_BUILD_TOOLS=OFF \ - -DLLVM_TABLEGEN=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-tblgen \ - -DLLVM_CONFIG_PATH=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config \ -" - -# The debug symbols are huge here (>2GB) so suppress them since they -# provide almost no value. If you really need them then override this -INHIBIT_PACKAGE_DEBUG_SPLIT = "1" - -export YOCTO_ALTERNATE_EXE_PATH = "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config" - -do_install:append () { - # we don't need any of this stuff to build Rust - rm -rf "${D}/usr/lib/cmake" -} - -PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liblto" - -# Add the extra locations to avoid the complaints about unpackaged files -FILES:${PN}-bugpointpasses = "${libdir}/llvm-rust/lib/BugpointPasses.so" -FILES:${PN}-llvmhello = "${libdir}/llvm-rust/lib/LLVMHello.so" -FILES:${PN}-liblto = "${libdir}/llvm-rust/lib/libLTO.so.*" -FILES:${PN}-staticdev =+ "${libdir}/llvm-rust/*/*.a" -FILES:${PN} += "${libdir}/libLLVM*.so.* ${libdir}/llvm-rust/lib/*.so.* ${libdir}/llvm-rust/bin" -FILES:${PN}-dev += "${datadir}/llvm ${libdir}/llvm-rust/lib/*.so ${libdir}/llvm-rust/include ${libdir}/llvm-rust/share ${libdir}/llvm-rust/lib/cmake" - -BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/rust/rust-llvm_1.66.0.bb +79 -1
85 lines not on this page · raw message
Signature · 1 line
2.39.0
PATCH v4 2/11 rust-llvm: Update LLVM_VERSION to match embedded version
In reply to Alex Kiernan---
(no changes since v1)
meta/recipes-devtools/rust/rust-llvm_1.66.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/rust/rust-llvm_1.66.0.bb +1 -1
@@ -4,7 +4,7 @@ HOMEPAGE = "http://www.rust-lang.org" # check src/llvm-project/llvm/CMakeLists.txt for llvm version in use # -LLVM_RELEASE = "14.0.5" +LLVM_RELEASE = "15.0.2" require rust-source.inc
Signature · 1 line
2.39.0
PATCH v4 3/11 packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
In reply to Alex Kiernan---
(no changes since v1)
.../packagegroups/packagegroup-rust-sdk-target.bb | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb
meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb new file +13 -0
@@ -0,0 +1,13 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +SUMMARY = "Target packages for the Rust SDK" + +inherit packagegroup + +RDEPENDS:${PN} = " \ + rust \ +"
Signature · 1 line
2.39.0
PATCH v4 4/11 packagegroup-core-sdk: Add SDK toolchain language selection support
In reply to Alex KiernanUse SDK_TOOLCHAIN_LANGS to allow different language support to be selected within SDKs. Initially supported options are rust and go.
---
(no changes since v3)
Changes in v3:
- Switch to packagegroup-core-sdk driven from TARGET_TOOLCHAIN_LANGS (which defaults to SDK_TOOLCHAIN_LANGS)
.../packagegroups/packagegroup-core-sdk.bb | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
meta/recipes-core/packagegroups/packagegroup-core-sdk.bb +9 -1
@@ -11,6 +11,11 @@ inherit packagegroup #PACKAGEFUNCS =+ 'generate_sdk_pkgs' +TARGET_TOOLCHAIN_LANGS ??= "${SDK_TOOLCHAIN_LANGS}" +TARGET_TOOLCHAIN_LANGS:remove:sdkmingw32 = "rust" +# libstd-rs doesn't build for mips n32 with compiler constraint errors +TARGET_TOOLCHAIN_LANGS:remove:mipsarchn32 = "rust" + RDEPENDS:packagegroup-core-sdk = "\ packagegroup-core-buildessential \ coreutils \
@@ -23,7 +28,10 @@ RDEPENDS:packagegroup-core-sdk = "\ less \ ldd \ file \ - tcl" + tcl \ + ${@bb.utils.contains('TARGET_TOOLCHAIN_LANGS', 'go', 'packagegroup-go-sdk-target', '', d)} \ + ${@bb.utils.contains('TARGET_TOOLCHAIN_LANGS', 'rust', 'packagegroup-rust-sdk-target', '', d)} \ +" SANITIZERS = "libasan-dev libubsan-dev" SANITIZERS:arc = ""
Signature · 1 line
2.39.0
PATCH v4 5/11 cargo: Merge .inc into .bb
In reply to Alex Kiernan---
(no changes since v1)
meta/recipes-devtools/cargo/cargo.inc | 69 -------------------- meta/recipes-devtools/cargo/cargo_1.66.0.bb | 71 ++++++++++++++++++++- 2 files changed, 69 insertions(+), 71 deletions(-) delete mode 100644 meta/recipes-devtools/cargo/cargo.inc
meta/recipes-devtools/cargo/cargo.inc deleted +0 -69
@@ -1,69 +0,0 @@ -SUMMARY ?= "Cargo, a package manager for Rust." -HOMEPAGE = "https://crates.io" -LICENSE = "MIT | Apache-2.0" -SECTION = "devel" - -DEPENDS = "openssl zlib curl ca-certificates libssh2" - -LIC_FILES_CHKSUM = " \ - file://LICENSE-MIT;md5=b377b220f43d747efdec40d69fcaa69d \ - file://LICENSE-APACHE;md5=71b224ca933f0676e26d5c2e2271331c \ - file://LICENSE-THIRD-PARTY;md5=f257ad009884cb88a3a87d6920e7180a \ -" - - -S = "${RUSTSRC}/src/tools/cargo" -CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor" -EXCLUDE_FROM_WORLD = "1" - -inherit cargo pkgconfig - -do_cargo_setup_snapshot () { - ${WORKDIR}/rust-snapshot-components/${CARGO_SNAPSHOT}/install.sh --prefix="${WORKDIR}/${CARGO_SNAPSHOT}" --disable-ldconfig - # Need to use uninative's loader if enabled/present since the library paths - # are used internally by rust and result in symbol mismatches if we don't - if [ ! -z "${UNINATIVE_LOADER}" -a -e "${UNINATIVE_LOADER}" ]; then - patchelf-uninative ${WORKDIR}/${CARGO_SNAPSHOT}/bin/cargo --set-interpreter ${UNINATIVE_LOADER} - fi -} - -addtask cargo_setup_snapshot after do_unpack before do_configure -do_cargo_setup_snapshot[dirs] += "${WORKDIR}/${CARGO_SNAPSHOT}" -do_cargo_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER" - - -do_compile:prepend () { - export RUSTC_BOOTSTRAP="1" -} - -do_install () { - install -d "${D}${bindir}" - install -m 755 "${B}/target/${CARGO_TARGET_SUBDIR}/cargo" "${D}${bindir}" -} - -do_install:append:class-nativesdk() { - # To quote the cargo docs, "Cargo also sets the dynamic library path when compiling - # and running binaries with commands like `cargo run` and `cargo test`". Sadly it - # sets to libdir but not base_libdir leading to symbol mismatches depending on the - # host OS. Fully set LD_LIBRARY_PATH to contain both to avoid this. - create_wrapper ${D}/${bindir}/cargo LD_LIBRARY_PATH=${libdir}:${base_libdir} -} - -# Disabled due to incompatibility with libgit2 0.28.x (https://github.com/rust-lang/git2-rs/issues/458, https://bugs.gentoo.org/707746#c1) -# as shipped by Yocto Dunfell. -# According to https://github.com/rust-lang/git2-rs/issues/458#issuecomment-522567539, there are no compatibility guarantees between -# libgit2-sys and arbitrary system libgit2 versions, so better keep this turned off. -#export LIBGIT2_SYS_USE_PKG_CONFIG = "1" - -# Needed for pkg-config to be used -export LIBSSH2_SYS_USE_PKG_CONFIG = "1" - -# When building cargo-native we don't have cargo-native to use and depend on, -# so we must use the locally set up snapshot to bootstrap the build. -BASEDEPENDS:remove:class-native = "cargo-native" -CARGO:class-native = "${WORKDIR}/${CARGO_SNAPSHOT}/bin/cargo" - -DEPENDS:append:class-nativesdk = " nativesdk-rust" -RUSTLIB:append:class-nativesdk = " -L ${STAGING_DIR_HOST}/${SDKPATHNATIVE}/usr/lib/rustlib/${RUST_HOST_SYS}/lib" - -
meta/recipes-devtools/cargo/cargo_1.66.0.bb +69 -2
75 lines not on this page · raw message
Signature · 1 line
2.39.0
PATCH v4 6/11 cargo: Extend DEBUG_PREFIX_MAP to cover vendor
In reply to Alex KiernanThe cargo build builds vendored libgit and curl, but these exist outside ${S} which DEBUG_PREFIX_MAP covers.
---
(no changes since v1)
meta/recipes-devtools/cargo/cargo_1.66.0.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-devtools/cargo/cargo_1.66.0.bb +2 -0
@@ -20,6 +20,8 @@ EXCLUDE_FROM_WORLD = "1" inherit cargo pkgconfig +DEBUG_PREFIX_MAP += "-fdebug-prefix-map=${RUSTSRC}/vendor=/usr/src/debug/${PN}/${EXTENDPE}${PV}-${PR}" + do_cargo_setup_snapshot () { ${WORKDIR}/rust-snapshot-components/${CARGO_SNAPSHOT}/install.sh --prefix="${WORKDIR}/${CARGO_SNAPSHOT}" --disable-ldconfig # Need to use uninative's loader if enabled/present since the library paths
Signature · 1 line
2.39.0
PATCH v4 7/11 rust: Merge .inc into .bb
In reply to Alex Kiernan---
(no changes since v1)
meta/recipes-devtools/rust/rust-target.inc | 10 - meta/recipes-devtools/rust/rust.inc | 223 --------------------- meta/recipes-devtools/rust/rust_1.66.0.bb | 218 +++++++++++++++++++- 3 files changed, 217 insertions(+), 234 deletions(-) delete mode 100644 meta/recipes-devtools/rust/rust-target.inc delete mode 100644 meta/recipes-devtools/rust/rust.inc
meta/recipes-devtools/rust/rust-target.inc deleted +0 -10
@@ -1,10 +0,0 @@ -require rust.inc - -DEPENDS += "rust-llvm (=${PV})" - -# Otherwise we'll depend on what we provide -INHIBIT_DEFAULT_RUST_DEPS:class-native = "1" -# We don't need to depend on gcc-native because yocto assumes it exists -PROVIDES:class-native = "virtual/${TARGET_PREFIX}rust" - -BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/rust/rust.inc deleted +0 -223
223 lines not on this page · raw message
meta/recipes-devtools/rust/rust_1.66.0.bb +217 -1
230 lines not on this page · raw message
Signature · 1 line
2.39.0
PATCH v4 8/11 rust: Move musl-x86 fix for `__stack_chk_fail_local` to rust-source
In reply to Alex KiernanAny consumer of rust-source (potentially) needs this, so move to rust-source.inc
---
Changes in v4:
- Fixup __stack_chk_fail_local in musl-x86
meta/recipes-devtools/rust/rust-source.inc | 5 +++++ meta/recipes-devtools/rust/rust_1.66.0.bb | 5 ----- 2 files changed, 5 insertions(+), 5 deletions(-)
meta/recipes-devtools/rust/rust-source.inc +5 -0
@@ -11,3 +11,8 @@ RUSTSRC = "${WORKDIR}/rustc-${RUST_VERSION}-src" UPSTREAM_CHECK_URI = "https://forge.rust-lang.org/infra/other-installation-methods.html" UPSTREAM_CHECK_REGEX = "rustc-(?P<pver>\d+(\.\d+)+)-src" + +# see recipes-devtools/gcc/gcc/0018-Add-ssp_nonshared-to-link-commandline-for-musl-targe.patch +# we need to link with ssp_nonshared on musl to avoid "undefined reference to `__stack_chk_fail_local'" +# when building MACHINE=qemux86 for musl +WRAPPER_TARGET_EXTRALD:libc-musl = "-lssp_nonshared"
meta/recipes-devtools/rust/rust_1.66.0.bb +0 -5
@@ -292,11 +292,6 @@ rust_do_install:class-target() { rm ${D}${libdir}/rustlib/manifest* } -# see recipes-devtools/gcc/gcc/0018-Add-ssp_nonshared-to-link-commandline-for-musl-targe.patch -# we need to link with ssp_nonshared on musl to avoid "undefined reference to `__stack_chk_fail_local'" -# when building MACHINE=qemux86 for musl -WRAPPER_TARGET_EXTRALD:libc-musl = "-lssp_nonshared" - RUSTLIB_DEP:class-nativesdk = "" # musl builds include libunwind.a
Signature · 1 line
2.39.0
PATCH v4 9/11 cargo: Override crossbeam-utils with 0.8.14
In reply to Alex KiernanUse cargo patching to replace crossbeam-utils with the semver compatible 0.8.14 rather than trying to patch our local changes. This corrects the ppc/mips builds.
---
Changes in v4:
- Add crossbeam atomics fixes for ppc/mips
meta/recipes-devtools/cargo/cargo_1.66.0.bb | 7 +++++++ 1 file changed, 7 insertions(+)
meta/recipes-devtools/cargo/cargo_1.66.0.bb +7 -0
@@ -14,6 +14,13 @@ LIC_FILES_CHKSUM = " \ require recipes-devtools/rust/rust-source.inc require recipes-devtools/rust/rust-snapshot.inc +# Reuse the path override capability we have to patch in a later version +# of crossbeam-utils which has the vendor misdetection fixes +# (https://github.com/crossbeam-rs/crossbeam/pull/922). This should be +# removed once the upstream source merges the change to include it. +SRC_URI:append:class-target = " crate://crates.io/crossbeam-utils/0.8.14" +EXTRA_OECARGO_PATHS:class-target = "${CARGO_HOME}/bitbake/crossbeam-utils-0.8.14" + S = "${RUSTSRC}/src/tools/cargo" CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor" EXCLUDE_FROM_WORLD = "1"
Signature · 1 line
2.39.0
PATCH v4 10/11 packagegroup-rust-sdk-target: Add cargo
In reply to Alex Kiernan---
(no changes since v1)
meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-core/packagegroups/packagegroup-rust-sdk-target.bb +1 -0
@@ -10,4 +10,5 @@ inherit packagegroup RDEPENDS:${PN} = " \ rust \ + cargo \ "
Signature · 1 line
2.39.0
PATCH v4 11/11 oeqa/runtime/rust: Add cargo test
In reply to Alex Kiernan---
(no changes since v1)
meta/lib/oeqa/runtime/cases/rust.py | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
meta/lib/oeqa/runtime/cases/rust.py +17 -0
@@ -20,6 +20,8 @@ class RustCompileTest(OERuntimeTestCase): def tearDown(cls): files = '/tmp/test.rs /tmp/test' cls.tc.target.run('rm %s' % files) + dirs = '/tmp/hello' + cls.tc.target.run('rm -r %s' % dirs) @OETestDepends(['ssh.SSHTest.test_ssh']) @OEHasPackage(['rust'])
@@ -32,6 +34,21 @@ class RustCompileTest(OERuntimeTestCase): msg = 'running compiled file failed, output: %s' % output self.assertEqual(status, 0, msg=msg) + @OETestDepends(['ssh.SSHTest.test_ssh']) + @OEHasPackage(['cargo']) + def test_cargo_compile(self): + status, output = self.target.run('cargo new /tmp/hello') + msg = 'cargo new failed, output: %s' % output + self.assertEqual(status, 0, msg=msg) + + status, output = self.target.run('cargo build --manifest-path=/tmp/hello/Cargo.toml') + msg = 'cargo build failed, output: %s' % output + self.assertEqual(status, 0, msg=msg) + + status, output = self.target.run('cargo run --manifest-path=/tmp/hello/Cargo.toml') + msg = 'running compiled file failed, output: %s' % output + self.assertEqual(status, 0, msg=msg) + class RustHelloworldTest(OERuntimeTestCase): @OETestDepends(['ssh.SSHTest.test_ssh']) @OEHasPackage(['rust-hello-world'])
Signature · 1 line
2.39.0
Quoted 19 lines · Alex Kiernan
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add
SDK_TOOLCHAIN_LANGSfor selection of target toolchains as part of packagegroup-core-sdk.Changes in v4:
- Fixup__stack_chk_fail_localin musl-x86
- Add crossbeam atomics fixes for ppc/mipsChanges in v3:
- Switch to packagegroup-core-sdk driven fromTARGET_TOOLCHAIN_LANGS(which defaults toSDK_TOOLCHAIN_LANGS)Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixes
Quoted 12 lines
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: UpdateLLVM_VERSIONto match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: ExtendDEBUG_PREFIX_MAPto cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for__stack_chk_fail_localto rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo test
Looks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.
Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.
Cheers,
Richard
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add
SDK_TOOLCHAIN_LANGSfor selection of target toolchains as part of packagegroup-core-sdk.Changes in v4:
- Fixup__stack_chk_fail_localin musl-x86
- Add crossbeam atomics fixes for ppc/mipsChanges in v3:
- Switch to packagegroup-core-sdk driven fromTARGET_TOOLCHAIN_LANGS(which defaults toSDK_TOOLCHAIN_LANGS)Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixesQuoted 12 lines
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: UpdateLLVM_VERSIONto match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: ExtendDEBUG_PREFIX_MAPto cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for__stack_chk_fail_localto rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo testLooks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.
Quoted 6 lines
Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.
Cc Sundeep who as been looking at it.
Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.
Can I get hold of the cargo packages generated here to have a poke at them?
Signature · 1 line
Alex Kiernan
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add
SDK_TOOLCHAIN_LANGSfor selection of target toolchains as part of packagegroup-core-sdk.Changes in v4:
- Fixup__stack_chk_fail_localin musl-x86
- Add crossbeam atomics fixes for ppc/mipsChanges in v3:
- Switch to packagegroup-core-sdk driven fromTARGET_TOOLCHAIN_LANGS(which defaults toSDK_TOOLCHAIN_LANGS)Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixesQuoted 12 lines
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: UpdateLLVM_VERSIONto match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: ExtendDEBUG_PREFIX_MAPto cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for__stack_chk_fail_localto rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo testLooks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.
Quoted 6 lines
Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.
Cc Sundeep who as been looking at it.
Quoted 8 lines
Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.
Can I get hold of the cargo packages generated here to have a poke at them?
Yes. The key line in the log is:
022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to /srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
i.e.
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
where you'll find both the package files and diffoscope output:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/
which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.
Cheers,
Richard
Quoted 70 lines · Richard Purdie
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add
SDK_TOOLCHAIN_LANGSfor selection of target toolchains as part of packagegroup-core-sdk.Changes in v4:
- Fixup__stack_chk_fail_localin musl-x86
- Add crossbeam atomics fixes for ppc/mipsChanges in v3:
- Switch to packagegroup-core-sdk driven fromTARGET_TOOLCHAIN_LANGS(which defaults toSDK_TOOLCHAIN_LANGS)Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixesQuoted 12 lines
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: UpdateLLVM_VERSIONto match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: ExtendDEBUG_PREFIX_MAPto cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for__stack_chk_fail_localto rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo testLooks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.
Quoted 6 lines
Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.
Cc Sundeep who as been looking at it.Quoted 8 lines
Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.
Can I get hold of the cargo packages generated here to have a poke at them?
Yes. The key line in the log is:
022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to
/srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efski.e.
Quoted 9 lines
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
where you'll find both the package files and diffoscope output:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/
which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.
Upstream has this ticket which, given how I did the patching for cargo, feels like the right kind of thing (rust has something similar out of the box):
https://github.com/rust-lang/rust/issues/98185
I've switched it around so that it uses local patching rather than cargo patching, which if it is this, seems like it might resolve it.
--
Alex Kiernan
Quoted 82 lines · Alex Kiernan
Quoted 70 lines · Richard Purdie
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add
SDK_TOOLCHAIN_LANGSfor selection of target toolchains as part of packagegroup-core-sdk.Changes in v4:
- Fixup__stack_chk_fail_localin musl-x86
- Add crossbeam atomics fixes for ppc/mipsChanges in v3:
- Switch to packagegroup-core-sdk driven fromTARGET_TOOLCHAIN_LANGS(which defaults toSDK_TOOLCHAIN_LANGS)Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixesQuoted 12 lines
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: UpdateLLVM_VERSIONto match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: ExtendDEBUG_PREFIX_MAPto cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for__stack_chk_fail_localto rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo testLooks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.
Quoted 6 lines
Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.
Cc Sundeep who as been looking at it.Quoted 8 lines
Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.
Can I get hold of the cargo packages generated here to have a poke at them?
Yes. The key line in the log is:
022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to
/srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efski.e.
Quoted 9 lines
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
where you'll find both the package files and diffoscope output:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/
which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.
Quoted 8 lines
Upstream has this ticket which, given how I did the patching for cargo, feels like the right kind of thing (rust has something similar out of the box):
https://github.com/rust-lang/rust/issues/98185
I've switched it around so that it uses local patching rather than cargo patching, which if it is this, seems like it might resolve it.
It was a different issue, thanks for tracking that down and fixing it!
The tests all passed so I merged the v5 series. Thanks for working through things, I think this makes our rust situation much more robust and should help future proof it :)
Cheers,
Richard
Quoted 88 lines · Richard Purdie
Quoted 86 lines · Alex Kiernan
Quoted 70 lines · Richard Purdie
Quoted 53 lines · Alex Kiernan
Quoted 44 lines · Richard Purdie
Quoted 19 lines · Alex Kiernan
This series adds runtime tests for rust and cargo, merges (most of) the include files into the base recipes and fixes target cargo builds.
Also add
SDK_TOOLCHAIN_LANGSfor selection of target toolchains as part of packagegroup-core-sdk.Changes in v4:
- Fixup__stack_chk_fail_localin musl-x86
- Add crossbeam atomics fixes for ppc/mipsChanges in v3:
- Switch to packagegroup-core-sdk driven fromTARGET_TOOLCHAIN_LANGS(which defaults toSDK_TOOLCHAIN_LANGS)Changes in v2:
- Drop rust.inc/rust.bb merge, not actually made use of in the change set
- Reorder so cargo test goes after build fixesQuoted 12 lines
Alex Kiernan (11):
rust-llvm: Merge .inc into .bb
rust-llvm: UpdateLLVM_VERSIONto match embedded version packagegroup-rust-sdk-target: Add Rust SDK target packagegroup
packagegroup-core-sdk: Add SDK toolchain language selection support
cargo: Merge .inc into .bb
cargo: ExtendDEBUG_PREFIX_MAPto cover vendor
rust: Merge .inc into .bb
rust: Move musl-x86 fix for__stack_chk_fail_localto rust-source
cargo: Override crossbeam-utils with 0.8.14 packagegroup-rust-sdk-target: Add cargo oeqa/runtime/rust: Add cargo testLooks much better thanks. There was one remaining issue which showed up in testing with a reproducibility issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2119/steps/12/logs/stdio
We did already have an open bug for a rust reproducibility issue and we excluded it from the tests until we could get to the bottom of it, I suspect cargo may have the same issue. It frustrates me a bit as it is the one reproducibility exclusion for core.
Quoted 6 lines
Ideally we would fix it but excluding it from the tests and noting it in the bug may be an option if it is the same issue. It may be simpler to reproduce and fix the issue with cargo too.
Cc Sundeep who as been looking at it.Quoted 8 lines
Just reading the ticket and the upstream ones... the upstream ticket is closed as remap path fixes it (which it looks like it does to me), there are build paths embedded in the build directory which don't get caught by that, but they're in thing like bootstrap which isn't packaged (which may or may not be an issue, but I'm suspecting not); it kinda feels like we're missing something.
Can I get hold of the cargo packages generated here to have a poke at them?
Yes. The key line in the log is:
022-12-26 11:56:30,555 - oe-selftest - INFO - Non-reproducible packages will be copied to
/srv/autobuilder/autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efski.e.
Quoted 9 lines
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk
where you'll find both the package files and diffoscope output:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20221226-0vk5efsk/packages/diff-html/
which seems to be saying the symbol table changed size which is unusual. I didn't look very deeply.
Upstream has this ticket which, given how I did the patching for cargo, feels like the right kind of thing (rust has something similar out of the box):
Quoted 7 lines
Quoted 4 lines
https://github.com/rust-lang/rust/issues/98185
I've switched it around so that it uses local patching rather than cargo patching, which if it is this, seems like it might resolve it.
It was a different issue, thanks for tracking that down and fixing it!
I'm not so sure - there's a .cargo/config.toml in the vendored rustc tarball which explicitly patches one of the vendored sources (it feels odd, but hey...), so I think we'll trip over this issue (assuming that's what the problem is).
Quoted 4 lines
The tests all passed so I merged the v5 series. Thanks for working through things, I think this makes our rust situation much more robust and should help future proof it :)
:)
Signature · 1 line
Alex Kiernan