rust: Avoid passing host-specific information to crates Strict Version Hash
PATCH v2 1/1 rust: Avoid passing host-specific information to crates Strict Version Hash
rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.
This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.
To avoid this, pass a fixed value instead of host-specific bits to the specified hash.
[YOCTO #15554]
--- ...ide-cfg-version-from-stable-crate-id.patch | 20 +++++++++++++++++++ meta/recipes-devtools/rust/rust-source.inc | 1 + 2 files changed, 21 insertions(+) create mode 100644 meta/recipes-devtools/rust/files/0003-rustc-span-add-oe-knob-to-elide-cfg-version-from-stable-crate-id.patch
meta/recipes-devtools/rust/files/0003-rustc-span-add-oe-knob-to-elide-cfg-version-from-stable-crate-id.patch new file +20 -0
@@ -0,0 +1,20 @@ +rust: Avoid passing host-dependent fingerprint to build artifacts + +To fix reproducibility issues, pass a fixed value instead of passing a +host-dependent fingerprint to the rust build artifacts via the Strict +Version Hash (SVH) + +Upstream-Status: Inappropriate [OE-specific reproducibility policy] +Signed-off-by: Alejandro Hernandez <[email protected]> +--- +--- a/compiler/rustc_span/src/def_id.rs ++++ b/compiler/rustc_span/src/def_id.rs +@@ -186,5 +186,3 @@ +- if let Some(val) = std::env::var_os("RUSTC_FORCE_RUSTC_VERSION") { +- hasher.write(val.to_string_lossy().into_owned().as_bytes()) +- } else { +- hasher.write(cfg_version.as_bytes()) +- } ++ // OE reproducible builds use a fixed value so host-varying bootstrap ++ // fingerprints do not perturb StableCrateId. ++ hasher.write(b"oe-stable-crate-id-no-cfg-version")
meta/recipes-devtools/rust/rust-source.inc +1 -0
@@ -6,6 +6,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} \ file://0001-src-core-build_steps-tool.rs-switch-off-lto-for-rust.patch;patchdir=${RUSTSRC} \ file://0002-Fix-rust-build-failure-with-unstable-options.patch;patchdir=${RUSTSRC} \ + file://0003-rustc-span-add-oe-knob-to-elide-cfg-version-from-stable-crate-id.patch;patchdir=${RUSTSRC} \ " SRC_URI[rust.sha256sum] = "0ed06fdaffd4722a7702e0b4eebfafc897ab8f513e8e1b247cdd7e5c6df6ded2"
Signature · 1 line
2.43.0
Quoted 7 lines · Alejandro Hernandez Samaniego via lists.openembedded.org
rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.
This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.
Quoted 7 lines
To avoid this, pass a fixed value instead of host-specific bits to the specified hash.
[YOCTO #15554]
Assisted-by AI - OpenAISigned-off-by Alejandro Hernandez <[email protected]>
Hi Alejandro,
This looks broken here:
ERROR: rust-native-1.97.1-r0 do_install: Execution of '/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/run.do_install.2682640' failed with exit code 1
...
ERROR: Logfile of failure stored in: /srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/log.do_install.2682640
...
| Building [==================> ] 275/348: object, schemars, tracing-s… | Compiling rustc_span v0.0.0 (/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work-shared/rust-source-1.97.1-r0/sources/rustc-1.97.1-src/compiler/rustc_span) | Building [==================> ] 276/348: object, schemars, tracing-s… | error: expected `;`, found `StableCrateId` | --> /rustc-dev/8bab26f4f68e0e26f0bb7960be334d5b520ea452/compiler/rustc_span/src/def_id.rs:200:59 | | | 200 | hasher.write(b"
https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/4510
Can you have a look at the issue?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 31 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 7 lines · Alejandro Hernandez Samaniego via lists.openembedded.org
rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.
This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.
Quoted 7 lines
To avoid this, pass a fixed value instead of host-specific bits to the specified hash.
[YOCTO #15554]
Assisted-by AI - OpenAISigned-off-by Alejandro Hernandez <[email protected]>Hi Alejandro,
This looks broken here:
ERROR: rust-native-1.97.1-r0
do_install: Execution of '/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/run.do_install.2682640' failed with exit code 1
...
ERROR: Logfile of failure stored in:/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/log.do_install.2682640
...| Building [==================> ] 275/348: object, schemars, tracing-s… | Compiling rustc_span v0.0.0 (/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work-shared/rust-source-1.97.1-r0/sources/rustc-1.97.1-src/compiler/rustc_span) | Building [==================> ] 276/348: object, schemars, tracing-s… | error: expected `;`, found `StableCrateId` | --> /rustc-dev/8bab26f4f68e0e26f0bb7960be334d5b520ea452/compiler/rustc_span/src/def_id.rs:200:59 | | | 200 | hasher.write(b"
Quoted 3 lines
https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/4510
Can you have a look at the issue?
I'll take a look
Quoted 12 lines
Thanks,
Mathieu-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#244262): https://lists.openembedded.org/g/openembedded-core/message/244262
Mute This Topic: https://lists.openembedded.org/mt/120913402/4354175
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 33 lines · Alejandro Hernandez Samaniego via lists.openembedded.org
Quoted 31 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 7 lines · Alejandro Hernandez Samaniego via lists.openembedded.org
rustc computes each crate's Strict Version Hash (SVH) using inputs that include the *stage0/stage1 bootstrap compiler* fingerprint, which in turn depends on the build host arch.
This eventually may cause sstate matches across architectures for artifacts that are actually different, causing autobuilder intermitent reproducibility issues.
Quoted 7 lines
To avoid this, pass a fixed value instead of host-specific bits to the specified hash.
[YOCTO #15554]
Assisted-by AI - OpenAISigned-off-by Alejandro Hernandez <[email protected]>Hi Alejandro,
This looks broken here:
ERROR: rust-native-1.97.1-r0
do_install: Execution of '/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/run.do_install.2682640' failed with exit code 1
...
ERROR: Logfile of failure stored in:/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/x86_64-linux/rust-native/1.97.1/temp/log.do_install.2682640
...| Building [==================> ] 275/348: object, schemars, tracing-s… | Compiling rustc_span v0.0.0 (/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work-shared/rust-source-1.97.1-r0/sources/rustc-1.97.1-src/compiler/rustc_span) | Building [==================> ] 276/348: object, schemars, tracing-s… | error: expected `;`, found `StableCrateId` | --> /rustc-dev/8bab26f4f68e0e26f0bb7960be334d5b520ea452/compiler/rustc_span/src/def_id.rs:200:59 | | | 200 | hasher.write(b"
Quoted 6 lines
Quoted 3 lines
https://autobuilder.yoctoproject.org/valkyrie/#/builders/4/builds/4510
Can you have a look at the issue?
I'll take a look
Looks like I made some mistakes when I rebased the patch onto the latest rust update that was just merged, sending v3
Quoted 17 lines
Quoted 8 lines
Thanks,
Mathieu-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#244272): https://lists.openembedded.org/g/openembedded-core/message/244272
Mute This Topic: https://lists.openembedded.org/mt/120913402/4354175
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-