Skip to contentSkip to conversation

rust: Avoid passing host-specific information to crates Strict Version Hash

PATCH Superseded Alejandro Hernandez 2 participants 4 messages Aug 24 – Aug 26 1 message not in the archive

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 rust: Avoid passing host-specific information to crates Strict Version Hash

rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.

This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.

To avoid this, pass a fixed value instead of host-specific bits to the specified hash.

[YOCTO #15554]

Assisted-by AI - OpenAI
---
 ...ide-cfg-version-from-stable-crate-id.patch | 20 +++++++++++++++++++
 meta/recipes-devtools/rust/rust-source.inc    |  1 +
 2 files changed, 21 insertions(+)
 create mode 100644 meta/recipes-devtools/rust/files/0003-rustc-span-add-oe-knob-to-elide-cfg-version-from-stable-crate-id.patch
meta/recipes-devtools/rust/files/0003-rustc-span-add-oe-knob-to-elide-cfg-version-from-stable-crate-id.patch new file +20 -0
new file mode 100644
index 0000000000..8409d49bbe
@@ -0,0 +1,20 @@
+rust: Avoid passing host-dependent fingerprint to build artifacts
+
+To fix reproducibility issues, pass a fixed value instead of passing a
+host-dependent fingerprint to the rust build artifacts via the Strict
+Version Hash (SVH)
+
+Upstream-Status: Inappropriate [OE-specific reproducibility policy]
+Signed-off-by: Alejandro Hernandez <[email protected]>
+---
+--- a/compiler/rustc_span/src/def_id.rs
++++ b/compiler/rustc_span/src/def_id.rs
+@@ -186,5 +186,3 @@
+-        if let Some(val) = std::env::var_os("RUSTC_FORCE_RUSTC_VERSION") {
+-            hasher.write(val.to_string_lossy().into_owned().as_bytes())
+-        } else {
+-            hasher.write(cfg_version.as_bytes())
+-        }
++        // OE reproducible builds use a fixed value so host-varying bootstrap
++        // fingerprints do not perturb StableCrateId.
++        hasher.write(b"oe-stable-crate-id-no-cfg-version")
meta/recipes-devtools/rust/rust-source.inc +1 -0
index 68f3009a7b..2bff69de3d 100644
@@ -6,6 +6,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
             file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} \
             file://0001-src-core-build_steps-tool.rs-switch-off-lto-for-rust.patch;patchdir=${RUSTSRC} \
             file://0002-Fix-rust-build-failure-with-unstable-options.patch;patchdir=${RUSTSRC} \
+            file://0003-rustc-span-add-oe-knob-to-elide-cfg-version-from-stable-crate-id.patch;patchdir=${RUSTSRC} \
 "
 SRC_URI[rust.sha256sum] = "0ed06fdaffd4722a7702e0b4eebfafc897ab8f513e8e1b247cdd7e5c6df6ded2"
 
Signature · 1 line
2.43.0
#1 lore raw
Quoted 7 lines · Alejandro Hernandez Samaniego via lists.openembedded.org

rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.

This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.

Quoted 7 lines

To avoid this, pass a fixed value instead of host-specific bits to the specified hash.

[YOCTO #15554]

Assisted-by AI - OpenAI

Hi Alejandro,

This looks broken here:

ERROR: rust-native-1.97.1-r0 do_install: Execution of '/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/run.do_install.2682640' failed with exit code 1
...
ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/log.do_install.2682640
...

|     Building [==================>      ] 275/348: object, schemars, tracing-s…
|    Compiling rustc_span v0.0.0 (/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work-shared/rust-source-1.97.1-r0/sources/rustc-1.97.1-src/compiler/rustc_span)
|     Building [==================>      ] 276/348: object, schemars, tracing-s…
| error: expected `;`, found `StableCrateId`
|    --> /rustc-dev/8bab26f4f68e0e26f0bb7960be334d5b520ea452/compiler/rustc_span/src/def_id.rs:200:59
|     |
| 200 |         hasher.write(b"

https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/4510

Can you have a look at the issue?

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#2 lore raw
Quoted 31 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 7 lines · Alejandro Hernandez Samaniego via lists.openembedded.org

rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.

This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.

Quoted 7 lines

To avoid this, pass a fixed value instead of host-specific bits to the specified hash.

[YOCTO #15554]

Assisted-by AI - OpenAI

Hi Alejandro,

This looks broken here:

ERROR: rust-native-1.97.1-r0 do_install: Execution of '/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/run.do_install.2682640' failed with exit code 1
...
ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/log.do_install.2682640
...

|     Building [==================>      ] 275/348: object, schemars, tracing-s…
|    Compiling rustc_span v0.0.0 (/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work-shared/rust-source-1.97.1-r0/sources/rustc-1.97.1-src/compiler/rustc_span)
|     Building [==================>      ] 276/348: object, schemars, tracing-s…
| error: expected `;`, found `StableCrateId`
|    --> /rustc-dev/8bab26f4f68e0e26f0bb7960be334d5b520ea452/compiler/rustc_span/src/def_id.rs:200:59
|     |
| 200 |         hasher.write(b"
Quoted 3 lines

https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/4510

Can you have a look at the issue?

I'll take a look

Quoted 12 lines

Thanks,
Mathieu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#244262): https://lists.openembedded.org/g/openembedded-core/message/244262
Mute This Topic: https://lists.openembedded.org/mt/120913402/4354175
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw
Quoted 33 lines · Alejandro Hernandez Samaniego via lists.openembedded.org
Quoted 31 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 7 lines · Alejandro Hernandez Samaniego via lists.openembedded.org

rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.

This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.

Quoted 7 lines

To avoid this, pass a fixed value instead of host-specific bits to the specified hash.

[YOCTO #15554]

Assisted-by AI - OpenAI

Hi Alejandro,

This looks broken here:

ERROR: rust-native-1.97.1-r0 do_install: Execution of '/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/run.do_install.2682640' failed with exit code 1
...
ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/log.do_install.2682640
...

|     Building [==================>      ] 275/348: object, schemars, tracing-s…
|    Compiling rustc_span v0.0.0 (/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work-shared/rust-source-1.97.1-r0/sources/rustc-1.97.1-src/compiler/rustc_span)
|     Building [==================>      ] 276/348: object, schemars, tracing-s…
| error: expected `;`, found `StableCrateId`
|    --> /rustc-dev/8bab26f4f68e0e26f0bb7960be334d5b520ea452/compiler/rustc_span/src/def_id.rs:200:59
|     |
| 200 |         hasher.write(b"
Quoted 6 lines
Quoted 3 lines

https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/4510

Can you have a look at the issue?

I'll take a look

Looks like I made some mistakes when I rebased the patch onto the latest rust update that was just merged, sending v3

Quoted 17 lines
Quoted 8 lines

Thanks,
Mathieu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#244272): https://lists.openembedded.org/g/openembedded-core/message/244272
Mute This Topic: https://lists.openembedded.org/mt/120913402/4354175
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-