Skip to contentSkip to conversation

rust: Fix reproducibility with shared source trees

PATCH In review [email protected] 2 participants 5 messages Jul 29

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/3 rust: Fix reproducibility with shared source trees

From: Deepesh Varatharajan <[email protected]>

Backport an upstream Rust commit to fix a reproducibility issue when Rust family recipes share the same source tree.

---
 ...ild-script-path-leakage-in-artifacts.patch | 28 +++++++++++++++++++
 meta/recipes-devtools/rust/rust-source.inc    |  1 +
 2 files changed, 29 insertions(+)
 create mode 100644 meta/recipes-devtools/rust/files/0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch
meta/recipes-devtools/rust/files/0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch new file +28 -0
new file mode 100644
index 0000000000..8c93624df7
@@ -0,0 +1,28 @@
+Backport a commit that remap OUT_DIR paths to fix build script path leakage in artifacts
+
+Upstream-Status: Backport [https://github.com/rust-lang/rust/commit/e743a4db62336027dcc1f1a755f4ca35f56b67df]
+
+Signed-off-by: Deepesh Varatharajan <[email protected]>
+
+diff --git a/src/bootstrap/src/core/builder/cargo.rs b/src/bootstrap/src/core/builder/cargo.rs
+index f785254..0264667 100644
+--- a/src/bootstrap/src/core/builder/cargo.rs
++++ b/src/bootstrap/src/core/builder/cargo.rs
+@@ -1077,6 +1077,8 @@ impl Builder<'_> {
+                         // rustc creates absolute paths (in part bc of the `rust-src` unremap
+                         // and for working directory) so let's remap the build directory as well.
+                         format!("{}={map_to}", self.build.src.display()),
++                        // remap OUT_DIR so they don't leak into artifacts.
++                        format!("{}={map_to}/out", self.build.out.display()),
+                     ]
+                     .join("\t");
+                     cargo.env("RUSTC_DEBUGINFO_MAP", map);
+@@ -1097,6 +1099,8 @@ impl Builder<'_> {
+                         // rustc creates absolute paths (in part bc of the `rust-src` unremap
+                         // and for working directory) so let's remap the build directory as well.
+                         format!("{}={map_to}", self.build.src.display()),
++                        // remap OUT_DIR so they don't leak into artifacts.
++                        format!("{}={map_to}/out", self.build.out.display()),
+                     ]
+                     .join("\t");
+                     cargo.env("RUSTC_DEBUGINFO_MAP", map);
meta/recipes-devtools/rust/rust-source.inc +1 -0
index a8d8de7165..5678e00671 100644
@@ -9,6 +9,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
             file://0002-Fix-rust-build-failure-with-unstable-options.patch;patchdir=${RUSTSRC} \
             file://0003-explicit-tail-calls-disable-two-tests.patch;patchdir=${RUSTSRC} \
             file://0004-explicit-tail-calls-ignore-tests-on-unsupported-llvm.patch;patchdir=${RUSTSRC} \
+            file://0005-remap-OUT_DIR-paths-to-fix-build-script-path-leakage-in-artifacts.patch;patchdir=${RUSTSRC} \
 "
 SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98"
 
Signature · 1 line
2.49.0
#1 lore raw

PATCH v4 2/3 rust: refactor source handling into a shared work-shared recipe

From: Deepesh Varatharajan <[email protected]>

[YOCTO #15808]

Consolidate Rust source management across all Rust recipes (rust, cargo, libstd-rs and their native/nativesdk variants) into a single shared source tree using the work-shared pattern.

Disk savings: a world build with multilibs enabled, together with nativesdk-rust and nativesdk-cargo, now shares a single ~4.2 GB rustc-src tree instead of maintaining up to 11 independent copies (~46.2 GB total).

Performance (clean core-image-sato build, qemux86-64):

Before: 68m53.507s
After: 65m46.159s
Improvement: 3m07.348s (~4.5%)

New files:

- rust-source_1.96.1.bb: A shared recipe that fetches, unpacks, and patches the rustc source tree into ${TMPDIR}/work-shared/rust-source-${PV}-${PR}. All build/install/packaging tasks are disabled; this recipe exists solely to provide a single copy of the source. Inherits allarch/nopackages, excludes itself from rm_work, and is hidden from world builds. Also defines RUST_BUILD_ARCH locally since this recipe does not inherit rust-common.bbclass but still needs the mapping for snapshot fetching via rust-snapshot.inc.

- common-source.inc: Included by consumer recipes (rust, cargo, libstd-rs) to disable their local do_fetch/do_unpack/do_patch tasks, clear SRC_URI, and add task dependencies on rust-source-${PV} for configure, licensing, and SPDX generation. Also provides is_work_shared_spdx() override for correct SPDX shared-workdir detection.

Modified files:

- rust-source.inc:
- Set UNPACKDIR to the work-shared location so all consumers share one source tree via RUSTSRC and S.

- rust_1.96.1.bb:
- Add 'require common-source.inc' to use the shared source tree.
- Set B = ${WORKDIR}/build to separate build artifacts from shared source.
- Set CARGO_HOME = ${WORKDIR}/cargo_home to isolate cargo caching.
- Set build-dir = "rust-build" in config.toml to distinguish Yocto's build directory (B), which holds config.toml, from the bootstrap's own build output directory. Without this, bootstrap defaults to "build/" inside B, creating an ambiguous build/build/ nesting.
- Use absolute ${S}/src/bootstrap/bootstrap.py path since cwd is now B.
- Update install functions to reference rust-build/ instead of build/.
- Update do_test_compile to use absolute ${S}/src/tools/ path.
- Change do_rust_setup_snapshot from 'after do_unpack' to depend on

    rust-source-${PV}:do_unpack (since local do_unpack is removed).

- cargo_1.96.1.bb:
- Add 'require common-source.inc' to use the shared source tree.
- Change do_cargo_setup_snapshot from 'after do_unpack' to depend on

    rust-source-${PV}:do_unpack.

- libstd-rs_1.96.1.bb:
- Add 'require common-source.inc' to use the shared source tree.

- rust-common.bbclass:
- Add a second --remap-path-prefix for ${TMPDIR}/work-shared so debug info from the shared source tree is correctly remapped.
- Retain RUST_BUILD_ARCH definition so that consumer recipes (rust, cargo, libstd-rs) which inherit this class can resolve snapshot directory names used by do_rust_setup_snapshot and do_cargo_setup_snapshot.

- lib/oeqa/selftest/cases/rust.py:
- Adapt the selftest to the build-dir/source-dir split: use B for build artifacts and RUSTSRC for source paths.
- Update remote-test-server copy path from build/ to rust-build/.
- Pass --build-dir and --config explicitly to bootstrap.py.

- conf/distro/include/maintainers.inc:
- Add maintainer entry for the new rust-source recipe.

License-Update: recipe file renames only, no license change

---
 meta/classes-recipe/rust-common.bbclass       |  4 +-
 meta/conf/distro/include/maintainers.inc      |  1 +
 meta/lib/oeqa/selftest/cases/rust.py          |  9 ++-
 meta/recipes-devtools/rust/cargo_1.96.1.bb    |  8 ++-
 meta/recipes-devtools/rust/common-source.inc  | 28 ++++++++++
 .../recipes-devtools/rust/libstd-rs_1.96.1.bb |  4 ++
 meta/recipes-devtools/rust/rust-source.inc    |  6 +-
 .../rust/rust-source_1.96.1.bb                | 56 +++++++++++++++++++
 meta/recipes-devtools/rust/rust_1.96.1.bb     | 31 +++++++---
 9 files changed, 132 insertions(+), 15 deletions(-)
 create mode 100644 meta/recipes-devtools/rust/common-source.inc
 create mode 100644 meta/recipes-devtools/rust/rust-source_1.96.1.bb
meta/classes-recipe/rust-common.bbclass +3 -1
index 057aeda67a..6bc42016d1 100644
@@ -14,7 +14,9 @@ FILES:${PN}-dev += "${rustlibdir}/*.rlib ${rustlibdir}/*.rmeta"
 FILES:${PN}-dbg += "${rustlibdir}/.debug"
 
 RUSTLIB ?= "-L ${STAGING_DIR_HOST}${rustlibdir}"
-RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR}"
+RUST_DEBUG_REMAP = "--remap-path-prefix=${WORKDIR}=${TARGET_DBGSRC_DIR} \
+                    --remap-path-prefix=${TMPDIR}/work-shared=${TARGET_DBGSRC_DIR} \
+"
 RUSTFLAGS += "${RUSTLIB} ${RUST_DEBUG_REMAP}"
 RUSTLIB_DEP ??= "libstd-rs"
 RUST_PANIC_STRATEGY ??= "unwind"
meta/conf/distro/include/maintainers.inc +1 -0
index e0edfdef7e..1565b18fec 100644
@@ -784,6 +784,7 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <[email protected]>"
 RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <[email protected]>"
 RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>"
+RECIPE_MAINTAINER:pn-rust-source-1.96.1 = "Deepesh Varatharajan <[email protected]>"
 RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <[email protected]>"
 RECIPE_MAINTAINER:pn-sassc = "Simone Weiß <[email protected]>"
 RECIPE_MAINTAINER:pn-sato-icon-theme = "Richard Purdie <[email protected]>"
meta/lib/oeqa/selftest/cases/rust.py +6 -3
index c961a8f652..4111d72044 100644
@@ -46,7 +46,8 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
         recipe = "rust"
         start_time = time.time()
         bitbake("{} -c test_compile".format(recipe))
-        builddir = get_bb_var("RUSTSRC", "rust")
+        builddir = get_bb_var("B", "rust")
+        sourcedir = get_bb_var("RUSTSRC", "rust")
         # build core-image-minimal with required packages
         default_installed_packages = ["libgcc", "libstdc++", "libatomic", "libgomp", "libzstd", "llvm", "openssl"]
         features = []
@@ -111,7 +112,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
             # Copy remote-test-server to image through scp
             host_sys = get_bb_var("RUST_BUILD_SYS", "rust")
             ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog, user="root")
-            ssh.copy_to(builddir + "/build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/")
+            ssh.copy_to(builddir + "/rust-build/" + host_sys + "/stage2-tools-bin/remote-test-server","~/")
             # Execute remote-test-server on image through background ssh
             command = '~/remote-test-server --bind 0.0.0.0:12345 -v'
             sshrun=subprocess.Popen(("ssh", '-o',  'UserKnownHostsFile=/dev/null', '-o',  'StrictHostKeyChecking=no', '-f', "root@%s" % qemu.ip, command), shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
@@ -131,8 +132,10 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, OEPTestResultTestCase):
             # PowerPC mac99 QEMU has 768MB RAM limit, so we need to minimize test binary sizes
             cmd = cmd + " export RUSTFLAGS='-C strip=debuginfo -Clink-arg=-lz -Clink-arg=-lzstd';"
             # Trigger testing.
+            # Run bootstrap from sourcedir with explicit --build-dir and --config
+            # pointing to the separate build directory.
             cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s" % (builddir, testargs, targetsys)
+            cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py --build-dir %s/rust-build --config %s/config.toml test %s --target %s" % (sourcedir, builddir, builddir, testargs, targetsys)
             retval = runCmd(cmd)
             end_time = time.time()
 
meta/recipes-devtools/rust/cargo_1.96.1.bb +7 -1
index adc1f0d331..16f8a76310 100644
@@ -13,6 +13,8 @@ LIC_FILES_CHKSUM = " \
 
 require rust-source.inc
 require rust-snapshot.inc
+# Use shared source tree from rust-source.bb; disables local fetch/unpack/patch
+require common-source.inc
 
 S = "${RUSTSRC}/src/tools/cargo"
 CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
@@ -21,6 +23,9 @@ CVE_PRODUCT = "rust-lang:cargo"
 
 inherit cargo pkgconfig
 
+# Explicit CARGO_HOME avoids cargo caching in the shared source tree
+CARGO_HOME = "${WORKDIR}/cargo_home"
+
 DEBUG_PREFIX_MAP += "-ffile-prefix-map=${RUSTSRC}/vendor=${TARGET_DBGSRC_DIR}"
 
 do_cargo_setup_snapshot () {
@@ -32,9 +37,10 @@ do_cargo_setup_snapshot () {
 	fi
 }
 
-addtask cargo_setup_snapshot after do_unpack before do_configure
+addtask cargo_setup_snapshot before do_configure
 do_cargo_setup_snapshot[dirs] += "${WORKDIR}/${CARGO_SNAPSHOT}"
 do_cargo_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER"
+do_cargo_setup_snapshot[depends] += "rust-source-${PV}:do_unpack"
 
 do_compile:prepend () {
 	export RUSTC_BOOTSTRAP="1"
meta/recipes-devtools/rust/common-source.inc new file +28 -0
new file mode 100644
index 0000000000..a0d4dc542c
@@ -0,0 +1,28 @@
+# common-source.inc - Disable source tasks for recipes using shared rust-source
+#
+# Recipes that include this file rely on rust-source-${PV} for fetching,
+# unpacking, and patching the Rust source tree. This avoids duplicating the
+# ~4.2 GB rustc-src tree per recipe variant (rust, cargo, libstd-rs and their
+# native/nativesdk counterparts).
+
+# Disable local fetch/unpack/patch since rust-source-${PV} handles them
+do_fetch() {
+        :
+}
+do_fetch[noexec] = "1"
+deltask do_unpack
+deltask do_patch
+
+# Clear SRC_URI to prevent fetcher warnings from stale URIs inherited via .inc
+SRC_URI = ""
+
+# Ensure the shared source is fully ready before dependent tasks run
+do_configure[depends] += "rust-source-${PV}:do_patch"
+do_populate_lic[depends] += "rust-source-${PV}:do_unpack"
+do_create_spdx[depends] += "rust-source-${PV}:do_patch"
+
+# The SPDX class uses is_work_shared_spdx() to detect shared workdirs.
+# Since we redirect S rather than WORKDIR, the default detection fails.
+# Override to always return True so SPDX processing works correctly.
+def is_work_shared_spdx(d):
+    return True
meta/recipes-devtools/rust/libstd-rs_1.96.1.bb +4 -0
index 1ecd6bff3f..939e5824b3 100644
@@ -5,6 +5,7 @@ LICENSE = "Unicode-3.0 AND (Apache-2.0 OR MIT)"
 LIC_FILES_CHKSUM = "file://../../COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
 
 require rust-source.inc
+require common-source.inc
 
 # The dummy crate named `sysroot` represents the standard library target.
 #
@@ -30,6 +31,9 @@ CARGO_FEATURES ?= "panic-unwind backtrace"
 CARGO_BUILD_FLAGS += "--features '${CARGO_FEATURES}'"
 CARGO_VENDORING_DIRECTORY = "${RUSTSRC}/vendor"
 
+# Explicit CARGO_HOME avoids cargo caching in the shared source tree
+CARGO_HOME = "${WORKDIR}/cargo_home"
+
 do_compile:prepend () {
     # For Rust 1.13.0 and newer
     export RUSTC_BOOTSTRAP="1"
meta/recipes-devtools/rust/rust-source.inc +5 -1
index 5678e00671..7b10eef9a6 100644
@@ -13,7 +13,11 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
 "
 SRC_URI[rust.sha256sum] = "77a6ff3003a4ad0cb00697b043c879e3e1a15d945b1a1f63818903bfc3fa8b98"
 
-RUSTSRC = "${UNPACKDIR}/rustc-${RUST_VERSION}-src"
+# Point UNPACKDIR into the work-shared location so rust-source.bb and all
+# consumers (via S/RUSTSRC) reference a single shared source tree.
+UNPACKDIR = "${TMPDIR}/work-shared/rust-source-${PV}-${PR}/sources"
+RUSTSRC = "${UNPACKDIR}/rustc-${PV}-src"
+S = "${RUSTSRC}"
 
 UPSTREAM_CHECK_URI = "https://forge.rust-lang.org/infra/other-installation-methods.html"
 UPSTREAM_CHECK_REGEX = "rustc-(?P<pver>\d+(\.\d+)+)-src"
meta/recipes-devtools/rust/rust-source_1.96.1.bb new file +56 -0

56 lines not on this page · raw message

meta/recipes-devtools/rust/rust_1.96.1.bb +22 -9
index fa8777826f..4bbe4ed0bd 100644
@@ -4,6 +4,10 @@ SECTION = "devel"
 LICENSE = "Unicode-3.0 AND (Apache-2.0 OR MIT)"
 LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
 
+require rust-source.inc
+require rust-snapshot.inc
+require common-source.inc
+
 inherit rust
 inherit cargo_common
 
@@ -24,6 +28,9 @@ INHIBIT_DEFAULT_RUST_DEPS:class-native = "1"
 PROVIDES:class-native = "virtual/${TARGET_PREFIX}rust"
 
 S = "${RUSTSRC}"
+# Separate build directory from the shared source tree so that multiple
+# variants (target, native, nativesdk) don't conflict in the source dir.
+B = "${WORKDIR}/build"
 
 # Use at your own risk, accepted values are stable, beta and nightly
 RUST_CHANNEL ?= "stable"
@@ -38,6 +45,9 @@ RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
 # own vendoring.
 CARGO_DISABLE_BITBAKE_VENDORING = "1"
 
+# Explicit CARGO_HOME avoids cargo caching in the shared source tree
+CARGO_HOME = "${WORKDIR}/cargo_home"
+
 setup_cargo_environment () {
     # The first step is to build bootstrap and some early stage tools,
     # these are build for the same target as the snapshot, e.g.
@@ -66,11 +76,11 @@ do_rust_setup_snapshot () {
         done
     fi
 }
-addtask rust_setup_snapshot after do_unpack before do_configure
+addtask rust_setup_snapshot before do_configure
 addtask do_test_compile after do_configure do_rust_gen_targets
 do_rust_setup_snapshot[dirs] += "${WORKDIR}/rust-snapshot"
 do_rust_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER"
-do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot"
+do_rust_setup_snapshot[depends] += "patchelf-native:do_populate_sysroot rust-source-${PV}:do_unpack"
 
 RUSTC_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/rustc"
 CARGO_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/cargo"
@@ -173,6 +183,12 @@ python do_configure() {
     # nothing about when trying to build some stage0 tools (like fabricate)
     config.set("build", "build", e(d.getVar("RUST_BUILD_SYS")))
 
+    # Yocto's B (${WORKDIR}/build) is where config.toml lives. Rust's
+    # bootstrap creates its own build output directory inside B; name it
+    # "rust-build" to avoid ambiguity with Yocto's build directory and
+    # prevent a confusing build/build/ nesting.
+    config.set("build", "build-dir", e("rust-build"))
+
     # [install]
     config.add_section("install")
     # ./x.py install doesn't have any notion of "destdir"
@@ -223,13 +239,10 @@ rust_runx () {
 
     oe_cargo_fix_env
 
-    python3 src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose
+    python3 ${S}/src/bootstrap/bootstrap.py ${@oe.utils.parallel_make_argument(d, '-j %d')} "$@" --verbose
 }
 rust_runx[vardepsexclude] += "PARALLEL_MAKE"
 
-require rust-source.inc
-require rust-snapshot.inc
-
 INSANE_SKIP:${PN}:class-native = "already-stripped"
 FILES:${PN} += "${libdir}/rustlib"
 FILES:${PN} += "${libdir}/*.so"
@@ -241,7 +254,7 @@ do_compile () {
 do_test_compile[dirs] = "${B}"
 do_test_compile () {
     replace_llvm_config_path
-    rust_runx build src/tools/remote-test-server --target "${RUST_TARGET_SYS}"
+    rust_runx build ${S}/src/tools/remote-test-server --target "${RUST_TARGET_SYS}"
 }
 
 ALLOW_EMPTY:${PN} = "1"
@@ -282,7 +295,7 @@ rust_do_install:class-nativesdk() {
 
     install -d ${D}${bindir}
     for i in cargo-clippy clippy-driver rustfmt; do
-        cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
+        cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
         patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i
     done
 
@@ -322,7 +335,7 @@ rust_do_install:class-target() {
 
     install -d ${D}${bindir}
     for i in ${EXTRA_TOOLS}; do
-        cp build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
+        cp rust-build/${RUST_BUILD_SYS}/stage2-tools/${RUST_HOST_SYS}/release/$i ${D}${bindir}
         patchelf --set-rpath "\$ORIGIN/../lib" ${D}${bindir}/$i
     done
 
Signature · 1 line
2.49.0
#2 lore raw

Patchtest results for [PATCH v4 2/3] rust: refactor source handling into a shared work-shared recipe

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/v4-2-3-rust-refactor-source-handling-into-a-shared-work-shared-recipe.patch

FAIL: test src uri left files: Patches not removed from tree. Remove them and amend the submitted mbox (test_metadata.TestMetadata.test_src_uri_left_files)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: pretest src uri left files (test_metadata.TestMetadata.pretest_src_uri_left_files)
PASS: test CVE check ignore (test_metadata.TestMetadata.test_cve_check_ignore)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test lic files chksum modified not mentioned (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
PASS: test lic files chksum presence (test_metadata.TestMetadata.test_lic_files_chksum_presence)
PASS: test license presence (test_metadata.TestMetadata.test_license_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test summary presence (test_metadata.TestMetadata.test_summary_presence)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

#3 lore raw

Re: Patchtest results for [PATCH v4 2/3] rust: refactor source handling into a shared work-shared recipe

Quoted 4 lines · Patchtest

FAIL: test src uri left files: Patches not removed from tree. Remove them and amend the submitted mbox (test_metadata.TestMetadata.test_src_uri_left_files)

Hi All,

his is a false positive. The patch files are not orphaned they are still referenced via SRC_URI in rust-source.inc, which is now required by the new rust-source_1.96.1.bb recipe.

The consumer recipes (rust, cargo, libstd-rs) include common-source.inc which clears their SRC_URI since they no longer fetch/unpack/patch themselves rust-source_1.96.1.bb handles that centrally.

Regards,
Deepesh

#4 lore raw

PATCH v4 3/3 rust: Move do_update_snapshot to rust-source

In reply to [email protected]

From: Deepesh Varatharajan <[email protected]>

Move do_update_snapshot from rust to rust-source, since rust-source now owns do_fetch/do_unpack/do_patch tasks after the shared source refactoring.

---
 .../rust/rust-source_1.96.1.bb                | 72 ++++++++++++++++++
 meta/recipes-devtools/rust/rust_1.96.1.bb     | 73 -------------------
 2 files changed, 72 insertions(+), 73 deletions(-)
meta/recipes-devtools/rust/rust-source_1.96.1.bb +72 -0
index e9163b4a97..5649c3e937 100644
@@ -54,3 +54,75 @@ B = "${WORKDIR}/build"
 RUST_BUILD_ARCH = "${@{'ppc': 'powerpc', 'ppc64': 'powerpc64', 'ppc64le': 'powerpc64le', 'riscv64': 'riscv64gc'}.get(d.getVar('BUILD_ARCH'), d.getVar('BUILD_ARCH'))}"
 
 EXCLUDE_FROM_WORLD = "1"
+
+addtask do_update_snapshot after do_patch
+do_update_snapshot[nostamp] = "1"
+
+# Run with `bitbake rust-source-${PV} -c update_snapshot` to update `rust-snapshot.inc`
+# with the checksums for the rust snapshot associated with this rustc-src tarball.
+python do_update_snapshot() {
+    import json
+    import re
+    import sys
+
+    from collections import defaultdict
+
+    key_value_pairs = {}
+    with open(os.path.join(d.getVar("S"), "src", "stage0")) as f:
+        for line in f:
+            # Skip empty lines or comments
+            if not line.strip() or line.startswith("#"):
+                continue
+            # Split the line into key and value using '=' as separator
+            match = re.match(r'(\S+)\s*=\s*(\S+)', line.strip())
+            if match:
+                key = match.group(1)
+                value = match.group(2)
+                key_value_pairs[key] = value
+    # Extract the required values from key_value_pairs
+    config_dist_server = key_value_pairs.get('dist_server', '')
+    compiler_date = key_value_pairs.get('compiler_date', '')
+    compiler_version = key_value_pairs.get('compiler_version', '')
+
+    src_uri = defaultdict(list)
+    # Assuming checksums_sha256 is now a key-value pair like: checksum_key = checksum_value
+    for k, v in key_value_pairs.items():
+        # Match the pattern for checksums
+        if "dist" in k and "tar.xz" in k:
+            m = re.search(f"dist/{compiler_date}/(?P<component>.*)-{compiler_version}-(?P<arch>.*)-unknown-linux-gnu\\.tar\\.xz", k)
+            if m:
+                component = m.group('component')
+                arch = m.group('arch')
+                src_uri[arch].append(f"SRC_URI[{component}-snapshot-{arch}.sha256sum] = \"{v}\"")
+    # Create the snapshot string with the extracted values
+    snapshot = """\
+## This is information on the rust-snapshot (binary) used to build our current release.
+## snapshot info is taken from rust/src/stage0
+## Rust is self-hosting and bootstraps itself with a pre-built previous version of itself.
+## The exact (previous) version that has been used is specified in the source tarball.
+## The version is replicated here.
+
+SNAPSHOT_VERSION = "%s"
+
+""" % compiler_version
+    # Add the checksum components to the snapshot
+    for arch, components in src_uri.items():
+        snapshot += "\n".join(components) + "\n\n"
+    # Add the additional snapshot URIs
+    snapshot += """\
+SRC_URI += " \\
+    ${RUST_DIST_SERVER}/dist/${RUST_STD_SNAPSHOT}.tar.xz;name=rust-std-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\
+    ${RUST_DIST_SERVER}/dist/${RUSTC_SNAPSHOT}.tar.xz;name=rustc-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\
+    ${RUST_DIST_SERVER}/dist/${CARGO_SNAPSHOT}.tar.xz;name=cargo-snapshot-${RUST_BUILD_ARCH};subdir=rust-snapshot-components \\
+"
+
+RUST_DIST_SERVER = "%s"
+
+RUST_STD_SNAPSHOT = "rust-std-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
+RUSTC_SNAPSHOT = "rustc-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
+CARGO_SNAPSHOT = "cargo-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
+""" % config_dist_server
+    # Write the updated snapshot information to the rust-snapshot.inc file
+    with open(os.path.join(d.getVar("THISDIR"), "rust-snapshot.inc"), "w") as f:
+        f.write(snapshot)
+}
meta/recipes-devtools/rust/rust_1.96.1.bb +0 -73

79 lines not on this page · raw message

Signature · 1 line
2.49.0