Skip to contentSkip to conversation

rust: Fix reproducibility with shared source trees

PATCH Superseded [email protected] 2 participants 4 messages Jul 21

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 rust: Fix reproducibility with shared source trees

From: Deepesh Varatharajan <[email protected]>

Backport an upstream Rust commit to fix a reproducibility issue when Rust family recipes share the same source tree.

---
 ...ild-script-path-leakage-in-artifacts.patch | 28 +++++++++++++++++++
 meta/recipes-devtools/rust/rust-source.inc    |  1 +
 2 files changed, 29 insertions(+)
 create mode 100644 meta/recipes-devtools/rust/files/0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch
meta/recipes-devtools/rust/files/0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch new file +28 -0
new file mode 100644
index 0000000000..8c93624df7
@@ -0,0 +1,28 @@
+Backport a commit that remap OUT_DIR paths to fix build script path leakage in artifacts
+
+Upstream-Status: Backport [https://github.com/rust-lang/rust/commit/e743a4db62336027dcc1f1a755f4ca35f56b67df]
+
+Signed-off-by: Deepesh Varatharajan <[email protected]>
+
+diff --git a/src/bootstrap/src/core/builder/cargo.rs b/src/bootstrap/src/core/builder/cargo.rs
+index f785254..0264667 100644
+--- a/src/bootstrap/src/core/builder/cargo.rs
++++ b/src/bootstrap/src/core/builder/cargo.rs
+@@ -1077,6 +1077,8 @@ impl Builder<'_> {
+                         // rustc creates absolute paths (in part bc of the `rust-src` unremap
+                         // and for working directory) so let's remap the build directory as well.
+                         format!("{}={map_to}", self.build.src.display()),
++                        // remap OUT_DIR so they don't leak into artifacts.
++                        format!("{}={map_to}/out", self.build.out.display()),
+                     ]
+                     .join("\t");
+                     cargo.env("RUSTC_DEBUGINFO_MAP", map);
+@@ -1097,6 +1099,8 @@ impl Builder<'_> {
+                         // rustc creates absolute paths (in part bc of the `rust-src` unremap
+                         // and for working directory) so let's remap the build directory as well.
+                         format!("{}={map_to}", self.build.src.display()),
++                        // remap OUT_DIR so they don't leak into artifacts.
++                        format!("{}={map_to}/out", self.build.out.display()),
+                     ]
+                     .join("\t");
+                     cargo.env("RUSTC_DEBUGINFO_MAP", map);
meta/recipes-devtools/rust/rust-source.inc +1 -0
index a8d8de7165..5678e00671 100644
@@ -9,6 +9,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
             file://0002-Fix-rust-build-failure-with-unstable-options.patch;patchdir=${RUSTSRC} \
             file://0003-explicit-tail-calls-disable-two-tests.patch;patchdir=${RUSTSRC} \
             file://0004-explicit-tail-calls-ignore-tests-on-unsupported-llvm.patch;patchdir=${RUSTSRC} \
+            file://0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch;patchdir=${RUSTSRC} \
 "
 SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98"
 
Signature · 1 line
2.49.0
#1 lore raw

PATCH 2/3 rust: refactor source handling into a shared work-shared recipe

From: Deepesh Varatharajan <[email protected]>

[YOCTO #15808]

Consolidate Rust source management across all Rust recipes (rust, cargo, libstd-rs and their native/nativesdk variants) into a single shared source tree using the work-shared pattern.

Disk savings: a world build with multilibs enabled, together with nativesdk-rust and nativesdk-cargo, now shares a single ~4.2 GB rustc-src tree instead of maintaining up to 11 independent copies (~46.2 GB total).

Performance (clean core-image-sato build, qemux86-64):

Before: 68m53.507s
After: 65m46.159s
Improvement: 3m07.348s (~4.5%)

New files:

- rust-source.bb: A shared recipe that fetches, unpacks, and patches the rustc source tree into ${TMPDIR}/work-shared/rust-source-${PV}-${PR}. All build/install/packaging tasks are disabled; this recipe exists solely to provide a single copy of the source. Inherits allarch/nopackages, excludes itself from rm_work, and is hidden from world builds. Also defines RUST_BUILD_ARCH locally since this recipe does not inherit rust-common.bbclass but still needs the mapping for snapshot fetching via rust-snapshot.inc.

- common-source.inc: Included by consumer recipes (rust, cargo, libstd-rs) to disable their local do_fetch/do_unpack/do_patch tasks, clear SRC_URI, and add task dependencies on rust-source-${PV} for configure, licensing, and SPDX generation. Also provides is_work_shared_spdx() override for correct SPDX shared-workdir detection.

Modified files:

- rust-source.inc:
- Replace RUST_VERSION with explicit MAJOR_VER/MINOR_VER/PATCH_VER and PV.
- Set UNPACKDIR to the work-shared location so all consumers share one source tree via RUSTSRC and S.

- rust_git.bb (renamed from rust_1.96.1.bb):
- Add 'require common-source.inc' to use the shared source tree.
- Set B = ${WORKDIR}/build to separate build artifacts from shared source.
- Set CARGO_HOME = ${WORKDIR}/cargo_home to isolate cargo caching.
- Set build-dir = "rust-build" in config.toml to distinguish Yocto's build directory (B), which holds config.toml, from the bootstrap's own build output directory. Without this, bootstrap defaults to "build/" inside B, creating an ambiguous build/build/ nesting.
- Use absolute ${S}/src/bootstrap/bootstrap.py path since cwd is now B.
- Update install functions to reference rust-build/ instead of build/.
- Update do_test_compile to use absolute ${S}/src/tools/ path.
- Change do_rust_setup_snapshot from 'after do_unpack' to depend on

    rust-source-${PV}:do_unpack (since local do_unpack is removed).

- cargo_git.bb (renamed from cargo_1.96.1.bb):
- Add 'require common-source.inc' to use the shared source tree.
- Change do_cargo_setup_snapshot from 'after do_unpack' to depend on

    rust-source-${PV}:do_unpack.

- libstd-rs_git.bb (renamed from libstd-rs_1.96.1.bb):
- Add 'require common-source.inc' to use the shared source tree.

- rust-cross-canadian_git.bb (renamed from rust-cross-canadian_1.96.1.bb):
- Rename only (version now comes from rust-source.inc via PV).

- rust-common.bbclass:
- Add a second --remap-path-prefix for ${TMPDIR}/work-shared so debug info from the shared source tree is correctly remapped.
- Retain RUST_BUILD_ARCH definition so that consumer recipes (rust, cargo, libstd-rs) which inherit this class can resolve snapshot directory names used by do_rust_setup_snapshot and do_cargo_setup_snapshot.

- lib/oeqa/selftest/cases/rust.py:
- Adapt the selftest to the build-dir/source-dir split: use B for build artifacts and RUSTSRC for source paths.
- Update remote-test-server copy path from build/ to rust-build/.
- Pass --build-dir and --config explicitly to bootstrap.py.

- conf/distro/include/maintainers.inc:
- Add maintainer entry for the new rust-source recipe.

---
 meta/classes-recipe/rust-common.bbclass       |  4 +-
 meta/conf/distro/include/maintainers.inc      |  1 +
 meta/lib/oeqa/selftest/cases/rust.py          |  9 ++-
 .../rust/{cargo_1.96.1.bb => cargo_git.bb}    |  8 ++-
 meta/recipes-devtools/rust/common-source.inc  | 28 ++++++++++
 .../{libstd-rs_1.96.1.bb => libstd-rs_git.bb} |  4 ++
 ...n_1.96.1.bb => rust-cross-canadian_git.bb} |  0
 meta/recipes-devtools/rust/rust-source.bb     | 56 +++++++++++++++++++
 meta/recipes-devtools/rust/rust-source.inc    | 16 +++++-
 .../rust/{rust_1.96.1.bb => rust_git.bb}      | 31 +++++++---
 10 files changed, 140 insertions(+), 17 deletions(-)
 rename meta/recipes-devtools/rust/{cargo_1.96.1.bb => cargo_git.bb} (91%)
 create mode 100644 meta/recipes-devtools/rust/common-source.inc
 rename meta/recipes-devtools/rust/{libstd-rs_1.96.1.bb => libstd-rs_git.bb} (92%)
 rename meta/recipes-devtools/rust/{rust-cross-canadian_1.96.1.bb => rust-cross-canadian_git.bb} (100%)
 create mode 100644 meta/recipes-devtools/rust/rust-source.bb
 rename meta/recipes-devtools/rust/{rust_1.96.1.bb => rust_git.bb} (93%)
meta/classes-recipe/rust-common.bbclass +3 -1
index 057aeda67a..6bc42016d1 100644
@@ -14,7 +14,9 @@ FILES:${PN}-dev += "${rustlibdir}/*.rlib ${rustlibdir}/*.rmeta"
 FILES:${PN}-dbg += "${rustlibdir}/.debug"
 
 RUSTLIB ?= "-L ${STAGING_DIR_HOST}${rustlibdir}"
-RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR}"
+RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR} \
+                    --remap-path-prefix=${TMPDIR}/work-shared=${TARGET_DBGSRC_DIR} \
+"
 RUSTFLAGS += "${RUSTLIB} ${RUST_DEBUG_REMAP}"
 RUSTLIB_DEP ??= "libstd-rs"
 RUST_PANIC_STRATEGY ??= "unwind"
meta/conf/distro/include/maintainers.inc +1 -0
index 4c6307086c..bfff5082c2 100644
@@ -784,6 +784,7 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <[email protected]>"
 RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <[email protected]>"
 RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>"
+RECIPE_MAINTAINER:pn-rust-source-1.96.1 = "Deepesh Varatharajan <[email protected]>"
 RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <[email protected]>"
 RECIPE_MAINTAINER:pn-sassc = "Simone Weiß <[email protected]>"
 RECIPE_MAINTAINER:pn-sato-icon-theme = "Richard Purdie <[email protected]>"
meta/lib/oeqa/selftest/cases/rust.py +6 -3
index c961a8f652..4111d72044 100644
@@ -46,7 +46,8 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
         recipe = "rust"
         start_time = time.time()
         bitbake("{} -c test_compile".format(recipe))
-        builddir = get_bb_var("RUSTSRC", "rust")
+        builddir = get_bb_var("B", "rust")
+        sourcedir = get_bb_var("RUSTSRC", "rust")
         # build core-image-minimal with required packages
         default_installed_packages = ["libgcc", "libstdc++", "libatomic", "libgomp", "libzstd", "llvm", "openssl"]
         features = []
@@ -111,7 +112,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
             # Copy remote-test-server to image through scp
             host_sys = get_bb_var("RUST_BUILD_SYS", "rust")
             ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user="root")
-            ssh.copy_to(builddir + "/build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/")
+            ssh.copy_to(builddir + "/rust-build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/")
             # Execute remote-test-server on image through background ssh
             command = '~/remote-test-server --bind 0.0.0.0:12345 -v'
             sshrun=subprocess.Popen(("ssh", '-o',  'UserKnownHostsFile=/dev/null', '-o',  'StrictHostKeyChecking=no', '-f', "root@%s" % qemu.ip, command), shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
@@ -131,8 +132,10 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
             # PowerPC mac99 QEMU has 768MB RAM limit, so we need to minimize test binary sizes
             cmd = cmd + " export RUSTFLAGS='-C strip=debuginfo -Clink-arg=-lz -Clink-arg=-lzstd';"
             # Trigger testing.
+            # Run bootstrap from sourcedir with explicit --build-dir and --config
+            # pointing to the separate build directory.
             cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s" % (builddir, testargs, targetsys)
+            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py --build-dir %s/rust-build --config %s/config.toml test %s --target %s" % (sourcedir, builddir, builddir, testargs, targetsys)
             retval = runCmd(cmd)
             end_time = time.time()
 
meta/recipes-devtools/rust/cargo_git.bb renamed from meta/recipes-devtools/rust/cargo_1.96.1.bb +7 -1
similarity index 91%
rename from meta/recipes-devtools/rust/cargo_1.96.1.bb
rename to meta/recipes-devtools/rust/cargo_git.bb
index f16688fc76..8047a2b255 100644
@@ -13,6 +13,8 @@ LIC_FILES_CHKSUM = " \
 
 require rust-source.inc
 require rust-snapshot.inc
+# Use shared source tree from rust-source.bb; disables local fetch/unpack/patch
+require common-source.inc
 
 S = "${RUSTSRC}/src/tools/cargo"
 CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
@@ -21,6 +23,9 @@ CVE_PRODUCT = "rust-lang:cargo"
 
 inherit cargo pkgconfig
 
+# Explicit CARGO_HOME avoids cargo caching in the shared source tree
+CARGO_HOME = "${WORKDIR}/cargo_home"
+
 DEBUG_PREFIX_MAP += "-ffile-prefix-map=${RUSTSRC}/vendor=${TARGET_DBGSRC_DIR}"
 
 do_cargo_setup_snapshot () {
@@ -32,9 +37,10 @@ do_cargo_setup_snapshot () {
 	fi
 }
 
-addtask cargo_setup_snapshot after do_unpack before do_configure
+addtask cargo_setup_snapshot before do_configure
 do_cargo_setup_snapshot[dirs] += "${WORKDIR}/${CARGO_SNAPSHOT}"
 do_cargo_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER"
+do_cargo_setup_snapshot[depends] += "rust-source-${PV}:do_unpack"
 
 do_compile:prepend () {
 	export RUSTC_BOOTSTRAP="1"
meta/recipes-devtools/rust/common-source.inc new file +28 -0
new file mode 100644
index 0000000000..a0d4dc542c
@@ -0,0 +1,28 @@
+# common-source.inc - Disable source tasks for recipes using shared rust-source
+#
+# Recipes that include this file rely on rust-source-${PV} for fetching,
+# unpacking, and patching the Rust source tree. This avoids duplicating the
+# ~4.2 GB rustc-src tree per recipe variant (rust, cargo, libstd-rs and their
+# native/nativesdk counterparts).
+
+# Disable local fetch/unpack/patch since rust-source-${PV} handles them
+do_fetch() {
+        :
+}
+do_fetch[noexec] = "1"
+deltask do_unpack
+deltask do_patch
+
+# Clear SRC_URI to prevent fetcher warnings from stale URIs inherited via .inc
+SRC_URI = ""
+
+# Ensure the shared source is fully ready before dependent tasks run
+do_configure[depends] += "rust-source-${PV}:do_patch"
+do_populate_lic[depends] += "rust-source-${PV}:do_unpack"
+do_create_spdx[depends] += "rust-source-${PV}:do_patch"
+
+# The SPDX class uses is_work_shared_spdx() to detect shared workdirs.
+# Since we redirect S rather than WORKDIR, the default detection fails.
+# Override to always return True so SPDX processing works correctly.
+def is_work_shared_spdx(d):
+    return True
meta/recipes-devtools/rust/libstd-rs_git.bb renamed from meta/recipes-devtools/rust/libstd-rs_1.96.1.bb +4 -0
similarity index 92%
rename from meta/recipes-devtools/rust/libstd-rs_1.96.1.bb
rename to meta/recipes-devtools/rust/libstd-rs_git.bb
index 2a5e977a5e..31d42a8525 100644
@@ -5,6 +5,7 @@ LICENSE = "(MIT | Apache-2.0) & Unicode-3.0"
 LIC_FILES_CHKSUM = "file://../../COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
 
 require rust-source.inc
+require common-source.inc
 
 # The dummy crate named `sysroot` represents the standard library target.
 #
@@ -30,6 +31,9 @@ CARGO_FEATURES ?= "panic-unwind backtrace"
 CARGO_BUILD_FLAGS += "--features '${CARGO_FEATURES}'"
 CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
 
+# Explicit CARGO_HOME avoids cargo caching in the shared source tree
+CARGO_HOME = "${WORKDIR}/cargo_home"
+
 do_compile:prepend () {
     # For Rust 1.13.0 and newer
     export RUSTC_BOOTSTRAP="1"
meta/recipes-devtools/rust/rust-cross-canadian_git.bb renamed from meta/recipes-devtools/rust/rust-cross-canadian_1.96.1.bb +0 -0
similarity index 100%
rename from meta/recipes-devtools/rust/rust-cross-canadian_1.96.1.bb
rename to meta/recipes-devtools/rust/rust-cross-canadian_git.bb
meta/recipes-devtools/rust/rust-source.bb new file +56 -0

56 lines not on this page · raw message

meta/recipes-devtools/rust/rust-source.inc +13 -3
index 5678e00671..49207a3947 100644
@@ -1,6 +1,12 @@
-RUST_VERSION ?= "${@d.getVar('PV').split('-')[0]}"
+# Version is split into MAJOR_VER and MINOR_VER for easy bumping and
+# to allow consistent use of PV across all consuming recipes.
+MAJOR_VER = "1"
+MINOR_VER = "96"
+PATCH_VER = "1"
 
-SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;name=rust \
+PV = "${MAJOR_VER}.${MINOR_VER}.${PATCH_VER}"
+
+SRC_URI += "https://static.rust-lang.org/dist/rustc-${PV}-src.tar.xz;name=rust \
             file://0001-Define-more-ioctl-codes-on-riscv32gc-unknown-linux-g.patch;patchdir=${RUSTSRC} \
             file://rust-oe-selftest.patch;patchdir=${RUSTSRC} \
             file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} \
@@ -13,7 +19,11 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
 "
 SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98"
 
-RUSTSRC = "${UNPACKDIR}/rustc-${RUST_VERSION}-src"
+# Point UNPACKDIR into the work-shared location so rust-source.bb and all
+# consumers (via S/RUSTSRC) reference a single shared source tree.
+UNPACKDIR = "${TMPDIR}/work-shared/rust-source-${PV}-${PR}/sources"
+RUSTSRC = "${UNPACKDIR}/rustc-${PV}-src"
+S = "${RUSTSRC}"
 
 UPSTREAM_CHECK_URI = "https://forge.rust-lang.org/infra/other-installation-methods.html"
 UPSTREAM_CHECK_REGEX = "rustc-(?P<pver>\d+(\.\d+)+)-src"
meta/recipes-devtools/rust/rust_git.bb renamed from meta/recipes-devtools/rust/rust_1.96.1.bb +22 -9
similarity index 93%
rename from meta/recipes-devtools/rust/rust_1.96.1.bb
rename to meta/recipes-devtools/rust/rust_git.bb
index 8b0f1d1459..da042ffaae 100644
@@ -4,6 +4,10 @@ SECTION = "devel"
 LICENSE = "(MIT | Apache-2.0) & Unicode-3.0"
 LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
 
+require rust-source.inc
+require rust-snapshot.inc
+require common-source.inc
+
 inherit rust
 inherit cargo_common
 
@@ -24,6 +28,9 @@ INHIBIT_DEFAULT_RUST_DEPS:class-native = "1"
 PROVIDES:class-native = "virtual/${TARGET_PREFIX}rust"
 
 S = "${RUSTSRC}"
+# Separate build directory from the shared source tree so that multiple
+# variants (target, native, nativesdk) don't conflict in the source dir.
+B = "${WORKDIR}/build"
 
 # Use at your own risk, accepted values are stable, beta and nightly
 RUST_CHANNEL ?= "stable"
@@ -38,6 +45,9 @@ RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
 # own vendoring.
 CARGO_DISABLE_BITBAKE_VENDORING = "1"
 
+# Explicit CARGO_HOME avoids cargo caching in the shared source tree
+CARGO_HOME = "${WORKDIR}/cargo_home"
+
 setup_cargo_environment () {
     # The first step is to build bootstrap and some early stage tools,
     # these are build for the same target as the snapshot, e.g.
@@ -66,11 +76,11 @@ do_rust_setup_snapshot () {
         done
     fi
 }
-addtask rust_setup_snapshot after do_unpack before do_configure
+addtask rust_setup_snapshot before do_configure
 addtask do_test_compile after do_configure do_rust_gen_targets
 do_rust_setup_snapshot[dirs] += "${WORKDIR}/rust-snapshot"
 do_rust_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER"
-do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot"
+do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot rust-source-${PV}:do_unpack"
 
 RUSTC_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/rustc"
 CARGO_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/cargo"
@@ -173,6 +183,12 @@ python do_configure() {
     # nothing about when trying to build some stage0 tools (like fabricate)
     config.set("build", "build", e(d.getVar("RUST_BUILD_SYS")))
 
+    # Yocto's B (${WORKDIR}/build) is where config.toml lives. Rust's
+    # bootstrap creates its own build output directory inside B; name it
+    # "rust-build" to avoid ambiguity with Yocto's build directory and
+    # prevent a confusing build/build/ nesting.
+    config.set("build", "build-dir", e("rust-build"))
+
     # [install]
     config.add_section("install")
     # ./x.py install doesn't have any notion of "destdir"
@@ -223,13 +239,10 @@ rust_runx () {
 
     oe_cargo_fix_env
 
-    python3 src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose
+    python3 ${S}/src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose
 }
 rust_runx[vardepsexclude] += "PARALLEL_MAKE"
 
-require rust-source.inc
-require rust-snapshot.inc
-
 INSANE_SKIP:${PN}:class-native = "already-stripped"
 FILES:${PN} += "${libdir}/rustlib"
 FILES:${PN} += "${libdir}/*.so"
@@ -241,7 +254,7 @@ do_compile () {
 do_test_compile[dirs] = "${B}"
 do_test_compile () {
     replace_llvm_config_path
-    rust_runx build src/tools/remote-test-server --target "${RUST_TARGET_SYS}"
+    rust_runx build ${S}/src/tools/remote-test-server --target "${RUST_TARGET_SYS}"
 }
 
 ALLOW_EMPTY:${PN} = "1"
@@ -282,7 +295,7 @@ rust_do_install:class-nativesdk() {
 
     install -d ${D}${bindir}
     for i in cargo-clippy clippy-driver rustfmt; do
-        cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
+        cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
         patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i
     done
 
@@ -322,7 +335,7 @@ rust_do_install:class-target() {
 
     install -d ${D}${bindir}
     for i in ${EXTRA_TOOLS}; do
-        cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
+        cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
         patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i
     done
 
Signature · 1 line
2.49.0
#2 lore raw

Patchtest results for [PATCH 2/3] rust: refactor source handling into a shared work-shared recipe

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/2-3-rust-refactor-source-handling-into-a-shared-work-shared-recipe.patch

FAIL: test lic files chksum modified not mentioned: LIC_FILES_CHKSUM changed without "License-Update:" tag and description in commit message (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test CVE check ignore (test_metadata.TestMetadata.test_cve_check_ignore)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

#3 lore raw

PATCH 3/3 rust: Move do_update_snapshot to rust-source

In reply to [email protected]

From: Deepesh Varatharajan <[email protected]>

Move do_update_snapshot from rust to rust-source, since rust-source now owns do_fetch/do_unpack/do_patch tasks after the shared source refactoring.

---
 meta/recipes-devtools/rust/rust-source.bb | 72 ++++++++++++++++++++++
 meta/recipes-devtools/rust/rust_git.bb    | 73 -----------------------
 2 files changed, 72 insertions(+), 73 deletions(-)
meta/recipes-devtools/rust/rust-source.bb +72 -0
index ba696984b3..4c29b8e3c7 100644
@@ -54,3 +54,75 @@ B = "${WORKDIR}/build"
 RUST_BUILD_ARCH = "${@{'ppc': 'powerpc', 'ppc64': 'powerpc64', 'ppc64le': 'powerpc64le', 'riscv64': 'riscv64gc'}.get(d.getVar('BUILD_ARCH'), d.getVar('BUILD_ARCH'))}"
 
 EXCLUDE_FROM_WORLD = "1"
+
+addtask do_update_snapshot after do_patch
+do_update_snapshot[nostamp] = "1"
+
+# Run with `bitbake rust-source-${PV} -c update_snapshot` to update `rust-snapshot.inc`
+# with the checksums for the rust snapshot associated with this rustc-src tarball.
+python do_update_snapshot() {
+    import json
+    import re
+    import sys
+
+    from collections import defaultdict
+
+    key_value_pairs = {}
+    with open(os.path.join(d.getVar("S"), "src", "stage0")) as f:
+        for line in f:
+            # Skip empty lines or comments
+            if not line.strip() or line.startswith("#"):
+                continue
+            # Split the line into key and value using '=' as separator
+            match = re.match(r'(\S+)\s*=\s*(\S+)', line.strip())
+            if match:
+                key = match.group(1)
+                value = match.group(2)
+                key_value_pairs[key] = value
+    # Extract the required values from key_value_pairs
+    config_dist_server = key_value_pairs.get('dist_server', '')
+    compiler_date = key_value_pairs.get('compiler_date', '')
+    compiler_version = key_value_pairs.get('compiler_version', '')
+
+    src_uri = defaultdict(list)
+    # Assuming checksums_sha256 is now a key-value pair like: checksum_key = checksum_value
+    for k, v in key_value_pairs.items():
+        # Match the pattern for checksums
+        if "dist" in k and "tar.xz" in k:
+            m = re.search(f"dist/{compiler_date}/(?P<component>.*)-{compiler_version}-(?P<arch>.*)-unknown-linux-gnu\\.tar\\.xz", k)
+            if m:
+                component = m.group('component')
+                arch = m.group('arch')
+                src_uri[arch].append(f"SRC_URI[{component}-snapshot-{arch}.sha256sum] = \"{v}\"")
+    # Create the snapshot string with the extracted values
+    snapshot = """\
+## This is information on the rust-snapshot (binary) used to build our current release.
+## snapshot info is taken from rust/src/stage0
+## Rust is self-hosting and bootstraps itself with a pre-built previous version of itself.
+## The exact (previous) version that has been used is specified in the source tarball.
+## The version is replicated here.
+
+SNAPSHOT_VERSION = "%s"
+
+""" % compiler_version
+    # Add the checksum components to the snapshot
+    for arch, components in src_uri.items():
+        snapshot += "\n".join(components) + "\n\n"
+    # Add the additional snapshot URIs
+    snapshot += """\
+SRC_URI += " \\
+    ${RUST_DIST_SERVER}/dist/${RUST_STD_SNAPSHOT}.tar.xz;name=rust-std-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\
+    ${RUST_DIST_SERVER}/dist/${RUSTC_SNAPSHOT}.tar.xz;name=rustc-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\
+    ${RUST_DIST_SERVER}/dist/${CARGO_SNAPSHOT}.tar.xz;name=cargo-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\
+"
+
+RUST_DIST_SERVER = "%s"
+
+RUST_STD_SNAPSHOT = "rust-std-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
+RUSTC_SNAPSHOT = "rustc-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
+CARGO_SNAPSHOT = "cargo-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
+""" % config_dist_server
+    # Write the updated snapshot information to the rust-snapshot.inc file
+    with open(os.path.join(d.getVar("THISDIR"), "rust-snapshot.inc"), "w") as f:
+        f.write(snapshot)
+}
meta/recipes-devtools/rust/rust_git.bb +0 -73

79 lines not on this page · raw message

Signature · 1 line
2.49.0