rust: Fix reproducibility with shared source trees
PATCH 1/3 rust: Fix reproducibility with shared source trees
From: Deepesh Varatharajan <[email protected]>
Backport an upstream Rust commit to fix a reproducibility issue when Rust family recipes share the same source tree.
--- ...ild-script-path-leakage-in-artifacts.patch | 28 +++++++++++++++++++ meta/recipes-devtools/rust/rust-source.inc | 1 + 2 files changed, 29 insertions(+) create mode 100644 meta/recipes-devtools/rust/files/0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch
meta/recipes-devtools/rust/files/0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch new file +28 -0
@@ -0,0 +1,28 @@ +Backport a commit that remap OUT_DIR paths to fix build script path leakage in artifacts + +Upstream-Status: Backport [https://github.com/rust-lang/rust/commit/e743a4db62336027dcc1f1a755f4ca35f56b67df] + +Signed-off-by: Deepesh Varatharajan <[email protected]> + +diff --git a/src/bootstrap/src/core/builder/cargo.rs b/src/bootstrap/src/core/builder/cargo.rs +index f785254..0264667 100644 +--- a/src/bootstrap/src/core/builder/cargo.rs ++++ b/src/bootstrap/src/core/builder/cargo.rs +@@ -1077,6 +1077,8 @@ impl Builder<'_> { + // rustc creates absolute paths (in part bc of the `rust-src` unremap + // and for working directory) so let's remap the build directory as well. + format!("{}={map_to}", self.build.src.display()), ++ // remap OUT_DIR so they don't leak into artifacts. ++ format!("{}={map_to}/out", self.build.out.display()), + ] + .join("\t"); + cargo.env("RUSTC_DEBUGINFO_MAP", map); +@@ -1097,6 +1099,8 @@ impl Builder<'_> { + // rustc creates absolute paths (in part bc of the `rust-src` unremap + // and for working directory) so let's remap the build directory as well. + format!("{}={map_to}", self.build.src.display()), ++ // remap OUT_DIR so they don't leak into artifacts. ++ format!("{}={map_to}/out", self.build.out.display()), + ] + .join("\t"); + cargo.env("RUSTC_DEBUGINFO_MAP", map);
meta/recipes-devtools/rust/rust-source.inc +1 -0
@@ -9,6 +9,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n file://0002-Fix-rust-build-failure-with-unstable-options.patch;patchdir=${RUSTSRC} \ file://0003-explicit-tail-calls-disable-two-tests.patch;patchdir=${RUSTSRC} \ file://0004-explicit-tail-calls-ignore-tests-on-unsupported-llvm.patch;patchdir=${RUSTSRC} \ + file://0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch;patchdir=${RUSTSRC} \ " SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98"
Signature · 1 line
2.49.0
PATCH 2/3 rust: refactor source handling into a shared work-shared recipe
From: Deepesh Varatharajan <[email protected]>
[YOCTO #15808]
Consolidate Rust source management across all Rust recipes (rust, cargo, libstd-rs and their native/nativesdk variants) into a single shared source tree using the work-shared pattern.
Disk savings: a world build with multilibs enabled, together with nativesdk-rust and nativesdk-cargo, now shares a single ~4.2 GB rustc-src tree instead of maintaining up to 11 independent copies (~46.2 GB total).
Performance (clean core-image-sato build, qemux86-64):
Before: 68m53.507s
After: 65m46.159s
Improvement: 3m07.348s (~4.5%)
New files:
- rust-source.bb: A shared recipe that fetches, unpacks, and patches the rustc source tree into ${TMPDIR}/work-shared/rust-source-${PV}-${PR}. All build/install/packaging tasks are disabled; this recipe exists solely to provide a single copy of the source. Inherits allarch/nopackages, excludes itself from rm_work, and is hidden from world builds. Also defines RUST_BUILD_ARCH locally since this recipe does not inherit rust-common.bbclass but still needs the mapping for snapshot fetching via rust-snapshot.inc.
- common-source.inc: Included by consumer recipes (rust, cargo, libstd-rs) to disable their local do_fetch/do_unpack/do_patch tasks, clear SRC_URI, and add task dependencies on rust-source-${PV} for configure, licensing, and SPDX generation. Also provides is_work_shared_spdx() override for correct SPDX shared-workdir detection.
Modified files:
- rust-source.inc:
- Replace RUST_VERSION with explicit MAJOR_VER/MINOR_VER/PATCH_VER and PV.
- Set UNPACKDIR to the work-shared location so all consumers share one source tree via RUSTSRC and S.
- rust_git.bb (renamed from rust_1.96.1.bb):
- Add 'require common-source.inc' to use the shared source tree.
- Set B = ${WORKDIR}/build to separate build artifacts from shared source.
- Set CARGO_HOME = ${WORKDIR}/cargo_home to isolate cargo caching.
- Set build-dir = "rust-build" in config.toml to distinguish Yocto's build directory (B), which holds config.toml, from the bootstrap's own build output directory. Without this, bootstrap defaults to "build/" inside B, creating an ambiguous build/build/ nesting.
- Use absolute ${S}/src/bootstrap/bootstrap.py path since cwd is now B.
- Update install functions to reference rust-build/ instead of build/.
- Update do_test_compile to use absolute ${S}/src/tools/ path.
- Change do_rust_setup_snapshot from 'after do_unpack' to depend on
rust-source-${PV}:do_unpack (since local do_unpack is removed).- cargo_git.bb (renamed from cargo_1.96.1.bb):
- Add 'require common-source.inc' to use the shared source tree.
- Change do_cargo_setup_snapshot from 'after do_unpack' to depend on
rust-source-${PV}:do_unpack.- libstd-rs_git.bb (renamed from libstd-rs_1.96.1.bb):
- Add 'require common-source.inc' to use the shared source tree.
- rust-cross-canadian_git.bb (renamed from rust-cross-canadian_1.96.1.bb):
- Rename only (version now comes from rust-source.inc via PV).
- rust-common.bbclass:
- Add a second --remap-path-prefix for ${TMPDIR}/work-shared so debug info from the shared source tree is correctly remapped.
- Retain RUST_BUILD_ARCH definition so that consumer recipes (rust, cargo, libstd-rs) which inherit this class can resolve snapshot directory names used by do_rust_setup_snapshot and do_cargo_setup_snapshot.
- lib/oeqa/selftest/cases/rust.py:
- Adapt the selftest to the build-dir/source-dir split: use B for build artifacts and RUSTSRC for source paths.
- Update remote-test-server copy path from build/ to rust-build/.
- Pass --build-dir and --config explicitly to bootstrap.py.
- conf/distro/include/maintainers.inc:
- Add maintainer entry for the new rust-source recipe.
---
meta/classes-recipe/rust-common.bbclass | 4 +-
meta/conf/distro/include/maintainers.inc | 1 +
meta/lib/oeqa/selftest/cases/rust.py | 9 ++-
.../rust/{cargo_1.96.1.bb => cargo_git.bb} | 8 ++-
meta/recipes-devtools/rust/common-source.inc | 28 ++++++++++
.../{libstd-rs_1.96.1.bb => libstd-rs_git.bb} | 4 ++
...n_1.96.1.bb => rust-cross-canadian_git.bb} | 0
meta/recipes-devtools/rust/rust-source.bb | 56 +++++++++++++++++++
meta/recipes-devtools/rust/rust-source.inc | 16 +++++-
.../rust/{rust_1.96.1.bb => rust_git.bb} | 31 +++++++---
10 files changed, 140 insertions(+), 17 deletions(-)
rename meta/recipes-devtools/rust/{cargo_1.96.1.bb => cargo_git.bb} (91%)
create mode 100644 meta/recipes-devtools/rust/common-source.inc
rename meta/recipes-devtools/rust/{libstd-rs_1.96.1.bb => libstd-rs_git.bb} (92%)
rename meta/recipes-devtools/rust/{rust-cross-canadian_1.96.1.bb => rust-cross-canadian_git.bb} (100%)
create mode 100644 meta/recipes-devtools/rust/rust-source.bb
rename meta/recipes-devtools/rust/{rust_1.96.1.bb => rust_git.bb} (93%)meta/classes-recipe/rust-common.bbclass +3 -1
@@ -14,7 +14,9 @@ FILES:${PN}-dev += "${rustlibdir}/*.rlib ${rustlibdir}/*.rmeta" FILES:${PN}-dbg += "${rustlibdir}/.debug" RUSTLIB ?= "-L ${STAGING_DIR_HOST}${rustlibdir}" -RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR}" +RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR} \ + --remap-path-prefix=${TMPDIR}/work-shared=${TARGET_DBGSRC_DIR} \ +" RUSTFLAGS += "${RUSTLIB} ${RUST_DEBUG_REMAP}" RUSTLIB_DEP ??= "libstd-rs" RUST_PANIC_STRATEGY ??= "unwind"
meta/conf/distro/include/maintainers.inc +1 -0
@@ -784,6 +784,7 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <[email protected]>" RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <[email protected]>" RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <[email protected]>" RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>" +RECIPE_MAINTAINER:pn-rust-source-1.96.1 = "Deepesh Varatharajan <[email protected]>" RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-sassc = "Simone Weiß <[email protected]>" RECIPE_MAINTAINER:pn-sato-icon-theme = "Richard Purdie <[email protected]>"
meta/lib/oeqa/selftest/cases/rust.py +6 -3
@@ -46,7 +46,8 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): recipe = "rust" start_time = time.time() bitbake("{} -c test_compile".format(recipe)) - builddir = get_bb_var("RUSTSRC", "rust") + builddir = get_bb_var("B", "rust") + sourcedir = get_bb_var("RUSTSRC", "rust") # build core-image-minimal with required packages default_installed_packages = ["libgcc", "libstdc++", "libatomic", "libgomp", "libzstd", "llvm", "openssl"] features = []
@@ -111,7 +112,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): # Copy remote-test-server to image through scp host_sys = get_bb_var("RUST_BUILD_SYS", "rust") ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user="root") - ssh.copy_to(builddir + "/build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/") + ssh.copy_to(builddir + "/rust-build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/") # Execute remote-test-server on image through background ssh command = '~/remote-test-server --bind 0.0.0.0:12345 -v' sshrun=subprocess.Popen(("ssh", '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-f', "root@%s" % qemu.ip, command), shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
@@ -131,8 +132,10 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase): # PowerPC mac99 QEMU has 768MB RAM limit, so we need to minimize test binary sizes cmd = cmd + " export RUSTFLAGS='-C strip=debuginfo -Clink-arg=-lz -Clink-arg=-lzstd';" # Trigger testing. + # Run bootstrap from sourcedir with explicit --build-dir and --config + # pointing to the separate build directory. cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s" % (builddir, testargs, targetsys) + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py --build-dir %s/rust-build --config %s/config.toml test %s --target %s" % (sourcedir, builddir, builddir, testargs, targetsys) retval = runCmd(cmd) end_time = time.time()
meta/recipes-devtools/rust/cargo_git.bb renamed from meta/recipes-devtools/rust/cargo_1.96.1.bb +7 -1
@@ -13,6 +13,8 @@ LIC_FILES_CHKSUM = " \ require rust-source.inc require rust-snapshot.inc +# Use shared source tree from rust-source.bb; disables local fetch/unpack/patch +require common-source.inc S = "${RUSTSRC}/src/tools/cargo" CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
@@ -21,6 +23,9 @@ CVE_PRODUCT = "rust-lang:cargo" inherit cargo pkgconfig +# Explicit CARGO_HOME avoids cargo caching in the shared source tree +CARGO_HOME = "${WORKDIR}/cargo_home" + DEBUG_PREFIX_MAP += "-ffile-prefix-map=${RUSTSRC}/vendor=${TARGET_DBGSRC_DIR}" do_cargo_setup_snapshot () {
@@ -32,9 +37,10 @@ do_cargo_setup_snapshot () { fi } -addtask cargo_setup_snapshot after do_unpack before do_configure +addtask cargo_setup_snapshot before do_configure do_cargo_setup_snapshot[dirs] += "${WORKDIR}/${CARGO_SNAPSHOT}" do_cargo_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER" +do_cargo_setup_snapshot[depends] += "rust-source-${PV}:do_unpack" do_compile:prepend () { export RUSTC_BOOTSTRAP="1"
meta/recipes-devtools/rust/common-source.inc new file +28 -0
@@ -0,0 +1,28 @@ +# common-source.inc - Disable source tasks for recipes using shared rust-source +# +# Recipes that include this file rely on rust-source-${PV} for fetching, +# unpacking, and patching the Rust source tree. This avoids duplicating the +# ~4.2 GB rustc-src tree per recipe variant (rust, cargo, libstd-rs and their +# native/nativesdk counterparts). + +# Disable local fetch/unpack/patch since rust-source-${PV} handles them +do_fetch() { + : +} +do_fetch[noexec] = "1" +deltask do_unpack +deltask do_patch + +# Clear SRC_URI to prevent fetcher warnings from stale URIs inherited via .inc +SRC_URI = "" + +# Ensure the shared source is fully ready before dependent tasks run +do_configure[depends] += "rust-source-${PV}:do_patch" +do_populate_lic[depends] += "rust-source-${PV}:do_unpack" +do_create_spdx[depends] += "rust-source-${PV}:do_patch" + +# The SPDX class uses is_work_shared_spdx() to detect shared workdirs. +# Since we redirect S rather than WORKDIR, the default detection fails. +# Override to always return True so SPDX processing works correctly. +def is_work_shared_spdx(d): + return True
meta/recipes-devtools/rust/libstd-rs_git.bb renamed from meta/recipes-devtools/rust/libstd-rs_1.96.1.bb +4 -0
@@ -5,6 +5,7 @@ LICENSE = "(MIT | Apache-2.0) & Unicode-3.0" LIC_FILES_CHKSUM = "file://../../COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc" require rust-source.inc +require common-source.inc # The dummy crate named `sysroot` represents the standard library target. #
@@ -30,6 +31,9 @@ CARGO_FEATURES ?= "panic-unwind backtrace" CARGO_BUILD_FLAGS += "--features '${CARGO_FEATURES}'" CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor" +# Explicit CARGO_HOME avoids cargo caching in the shared source tree +CARGO_HOME = "${WORKDIR}/cargo_home" + do_compile:prepend () { # For Rust 1.13.0 and newer export RUSTC_BOOTSTRAP="1"
meta/recipes-devtools/rust/rust-cross-canadian_git.bb renamed from meta/recipes-devtools/rust/rust-cross-canadian_1.96.1.bb +0 -0
meta/recipes-devtools/rust/rust-source.bb new file +56 -0
56 lines not on this page · raw message
meta/recipes-devtools/rust/rust-source.inc +13 -3
@@ -1,6 +1,12 @@ -RUST_VERSION ?= "${@d.getVar('PV').split('-')[0]}" +# Version is split into MAJOR_VER and MINOR_VER for easy bumping and +# to allow consistent use of PV across all consuming recipes. +MAJOR_VER = "1" +MINOR_VER = "96" +PATCH_VER = "1" -SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;name=rust \ +PV = "${MAJOR_VER}.${MINOR_VER}.${PATCH_VER}" + +SRC_URI += "https://static.rust-lang.org/dist/rustc-${PV}-src.tar.xz;name=rust \ file://0001-Define-more-ioctl-codes-on-riscv32gc-unknown-linux-g.patch;patchdir=${RUSTSRC} \ file://rust-oe-selftest.patch;patchdir=${RUSTSRC} \ file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} \
@@ -13,7 +19,11 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n " SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98" -RUSTSRC = "${UNPACKDIR}/rustc-${RUST_VERSION}-src" +# Point UNPACKDIR into the work-shared location so rust-source.bb and all +# consumers (via S/RUSTSRC) reference a single shared source tree. +UNPACKDIR = "${TMPDIR}/work-shared/rust-source-${PV}-${PR}/sources" +RUSTSRC = "${UNPACKDIR}/rustc-${PV}-src" +S = "${RUSTSRC}" UPSTREAM_CHECK_URI = "https://forge.rust-lang.org/infra/other-installation-methods.html" UPSTREAM_CHECK_REGEX = "rustc-(?P<pver>\d+(\.\d+)+)-src"
meta/recipes-devtools/rust/rust_git.bb renamed from meta/recipes-devtools/rust/rust_1.96.1.bb +22 -9
@@ -4,6 +4,10 @@ SECTION = "devel" LICENSE = "(MIT | Apache-2.0) & Unicode-3.0" LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc" +require rust-source.inc +require rust-snapshot.inc +require common-source.inc + inherit rust inherit cargo_common
@@ -24,6 +28,9 @@ INHIBIT_DEFAULT_RUST_DEPS:class-native = "1" PROVIDES:class-native = "virtual/${TARGET_PREFIX}rust" S = "${RUSTSRC}" +# Separate build directory from the shared source tree so that multiple +# variants (target, native, nativesdk) don't conflict in the source dir. +B = "${WORKDIR}/build" # Use at your own risk, accepted values are stable, beta and nightly RUST_CHANNEL ?= "stable"
@@ -38,6 +45,9 @@ RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config" # own vendoring. CARGO_DISABLE_BITBAKE_VENDORING = "1" +# Explicit CARGO_HOME avoids cargo caching in the shared source tree +CARGO_HOME = "${WORKDIR}/cargo_home" + setup_cargo_environment () { # The first step is to build bootstrap and some early stage tools, # these are build for the same target as the snapshot, e.g.
@@ -66,11 +76,11 @@ do_rust_setup_snapshot () { done fi } -addtask rust_setup_snapshot after do_unpack before do_configure +addtask rust_setup_snapshot before do_configure addtask do_test_compile after do_configure do_rust_gen_targets do_rust_setup_snapshot[dirs] += "${WORKDIR}/rust-snapshot" do_rust_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER" -do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot" +do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot rust-source-${PV}:do_unpack" RUSTC_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/rustc" CARGO_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/cargo"
@@ -173,6 +183,12 @@ python do_configure() { # nothing about when trying to build some stage0 tools (like fabricate) config.set("build", "build", e(d.getVar("RUST_BUILD_SYS"))) + # Yocto's B (${WORKDIR}/build) is where config.toml lives. Rust's + # bootstrap creates its own build output directory inside B; name it + # "rust-build" to avoid ambiguity with Yocto's build directory and + # prevent a confusing build/build/ nesting. + config.set("build", "build-dir", e("rust-build")) + # [install] config.add_section("install") # ./x.py install doesn't have any notion of "destdir"
@@ -223,13 +239,10 @@ rust_runx () { oe_cargo_fix_env - python3 src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose + python3 ${S}/src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose } rust_runx[vardepsexclude] += "PARALLEL_MAKE" -require rust-source.inc -require rust-snapshot.inc - INSANE_SKIP:${PN}:class-native = "already-stripped" FILES:${PN} += "${libdir}/rustlib" FILES:${PN} += "${libdir}/*.so"
@@ -241,7 +254,7 @@ do_compile () { do_test_compile[dirs] = "${B}" do_test_compile () { replace_llvm_config_path - rust_runx build src/tools/remote-test-server --target "${RUST_TARGET_SYS}" + rust_runx build ${S}/src/tools/remote-test-server --target "${RUST_TARGET_SYS}" } ALLOW_EMPTY:${PN} = "1"
@@ -282,7 +295,7 @@ rust_do_install:class-nativesdk() { install -d ${D}${bindir} for i in cargo-clippy clippy-driver rustfmt; do - cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} + cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i done
@@ -322,7 +335,7 @@ rust_do_install:class-target() { install -d ${D}${bindir} for i in ${EXTRA_TOOLS}; do - cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} + cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir} patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i done
Signature · 1 line
2.49.0
Patchtest results for [PATCH 2/3] rust: refactor source handling into a shared work-shared recipe
Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:
---
Testing patch /home/patchtest/share/mboxes/2-3-rust-refactor-source-handling-into-a-shared-work-shared-recipe.patch
FAIL: test lic files chksum modified not mentioned: LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test CVE check ignore (test_metadata.TestMetadata.test_cve_check_ignore)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
---
Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!
PATCH 3/3 rust: Move do_update_snapshot to rust-source
In reply to [email protected]From: Deepesh Varatharajan <[email protected]>
Move do_update_snapshot from rust to rust-source, since rust-source now owns do_fetch/do_unpack/do_patch tasks after the shared source refactoring.
--- meta/recipes-devtools/rust/rust-source.bb | 72 ++++++++++++++++++++++ meta/recipes-devtools/rust/rust_git.bb | 73 ----------------------- 2 files changed, 72 insertions(+), 73 deletions(-)
meta/recipes-devtools/rust/rust-source.bb +72 -0
@@ -54,3 +54,75 @@ B = "${WORKDIR}/build" RUST_BUILD_ARCH = "${@{'ppc': 'powerpc', 'ppc64': 'powerpc64', 'ppc64le': 'powerpc64le', 'riscv64': 'riscv64gc'}.get(d.getVar('BUILD_ARCH'), d.getVar('BUILD_ARCH'))}" EXCLUDE_FROM_WORLD = "1" + +addtask do_update_snapshot after do_patch +do_update_snapshot[nostamp] = "1" + +# Run with `bitbake rust-source-${PV} -c update_snapshot` to update `rust-snapshot.inc` +# with the checksums for the rust snapshot associated with this rustc-src tarball. +python do_update_snapshot() { + import json + import re + import sys + + from collections import defaultdict + + key_value_pairs = {} + with open(os.path.join(d.getVar("S"), "src", "stage0")) as f: + for line in f: + # Skip empty lines or comments + if not line.strip() or line.startswith("#"): + continue + # Split the line into key and value using '=' as separator + match = re.match(r'(\S+)\s*=\s*(\S+)', line.strip()) + if match: + key = match.group(1) + value = match.group(2) + key_value_pairs[key] = value + # Extract the required values from key_value_pairs + config_dist_server = key_value_pairs.get('dist_server', '') + compiler_date = key_value_pairs.get('compiler_date', '') + compiler_version = key_value_pairs.get('compiler_version', '') + + src_uri = defaultdict(list) + # Assuming checksums_sha256 is now a key-value pair like: checksum_key = checksum_value + for k, v in key_value_pairs.items(): + # Match the pattern for checksums + if "dist" in k and "tar.xz" in k: + m = re.search(f"dist/{compiler_date}/(?P<component>.*)-{compiler_version}-(?P<arch>.*)-unknown-linux-gnu\\.tar\\.xz", k) + if m: + component = m.group('component') + arch = m.group('arch') + src_uri[arch].append(f"SRC_URI[{component}-snapshot-{arch}.sha256sum] = \"{v}\"") + # Create the snapshot string with the extracted values + snapshot = """\ +## This is information on the rust-snapshot (binary) used to build our current release. +## snapshot info is taken from rust/src/stage0 +## Rust is self-hosting and bootstraps itself with a pre-built previous version of itself. +## The exact (previous) version that has been used is specified in the source tarball. +## The version is replicated here. + +SNAPSHOT_VERSION = "%s" + +""" % compiler_version + # Add the checksum components to the snapshot + for arch, components in src_uri.items(): + snapshot += "\n".join(components) + "\n\n" + # Add the additional snapshot URIs + snapshot += """\ +SRC_URI += " \\ + ${RUST_DIST_SERVER}/dist/${RUST_STD_SNAPSHOT}.tar.xz;name=rust-std-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\ + ${RUST_DIST_SERVER}/dist/${RUSTC_SNAPSHOT}.tar.xz;name=rustc-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\ + ${RUST_DIST_SERVER}/dist/${CARGO_SNAPSHOT}.tar.xz;name=cargo-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\ +" + +RUST_DIST_SERVER = "%s" + +RUST_STD_SNAPSHOT = "rust-std-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu" +RUSTC_SNAPSHOT = "rustc-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu" +CARGO_SNAPSHOT = "cargo-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu" +""" % config_dist_server + # Write the updated snapshot information to the rust-snapshot.inc file + with open(os.path.join(d.getVar("THISDIR"), "rust-snapshot.inc"), "w") as f: + f.write(snapshot) +}
meta/recipes-devtools/rust/rust_git.bb +0 -73
79 lines not on this page · raw message
Signature · 1 line
2.49.0