Use frozen flag for cargo build
cover Use frozen flag for cargo build
From: Frederic Martinsons <[email protected]>
This patch series force the usage of --frozen instead of --offline during cargo build. This has the advantage to be sure that Cargo.lock file will not be modified. Moreover, raise a clear error when Cargo.lock is not present.
For this to work, we must modify ourself the Cargo.lock before building in case there are git repo that have been patched.
The last commit was for removing rust-hello-world, patch to the documentation project will follow.
The following changes since commit 692e414aed5313ff275b69e93179aa7c559700f3:
ffmpeg: Fix wrong code found with gas/2.41 (2023-08-01 09:49:14 +0100)
are available in the Git repository at:
https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
Frederic Martinsons (3):
cargo.bbclass: Use --frozen flag for cargo operations cargo_common.bbclass: Handle Cargo.lock modifications for git
dependencies
drop rust-hello-world recipe
meta/classes-recipe/cargo.bbclass | 7 +++- meta/classes-recipe/cargo_common.bbclass | 41 +++++++++++++++++++ meta/conf/distro/include/maintainers.inc | 1 - meta/lib/oeqa/runtime/cases/rust.py | 12 ------ .../packagegroup-core-tools-testapps.bb | 3 -- .../rust-hello-world/0001-enable-LTO.patch | 24 ----------- .../rust-example/rust-hello-world_git.bb | 19 --------- 7 files changed, 47 insertions(+), 60 deletions(-) delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world_git.bb
Signature · 1 line
2.34.1
PATCH 1/3 cargo.bbclass: Use --frozen flag for cargo operations
From: Frederic Martinsons <[email protected]>
It supersed the --offline flag and guarantee that Cargo.lock file will not be modified during the build.
--- meta/classes-recipe/cargo.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
meta/classes-recipe/cargo.bbclass +6 -1
@@ -39,7 +39,12 @@ MANIFEST_PATH ??= "${S}/${CARGO_SRC_DIR}/Cargo.toml" RUSTFLAGS ??= "" BUILD_MODE = "${@['--release', ''][d.getVar('DEBUG_BUILD') == '1']}" -CARGO_BUILD_FLAGS = "-v --offline --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}" +# --frozen flag will prevent network access (which is required since only +# the do_fetch step is authorized to access network) +# and will require an up to date Cargo.lock file. +# This force the package being built to already ship a Cargo.lock, in the end +# this is what we want, at least, for reproducibility of the build. +CARGO_BUILD_FLAGS = "-v --frozen --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}" # This is based on the content of CARGO_BUILD_FLAGS and generally will need to # change if CARGO_BUILD_FLAGS changes.
Signature · 1 line
2.34.1
PATCH 2/3 cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies
In reply to [email protected]From: Frederic Martinsons <[email protected]>
Now we use --frozen, Cargo.lock cannot be modified by cargo build. These patched git dependencies requires that the git url is removed from Cargo.lock.
Fixes #15104
--- meta/classes-recipe/cargo_common.bbclass | 41 ++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
meta/classes-recipe/cargo_common.bbclass +41 -0
@@ -117,6 +117,8 @@ cargo_common_do_configure () { } python cargo_common_do_patch_paths() { + import shutil + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") if not os.path.exists(cargo_config): return
@@ -146,6 +148,45 @@ python cargo_common_do_patch_paths() { print('\n[patch."%s"]' % k, file=config) for name in v: print(name, file=config) + + if not patches: + return + + # Cargo.lock file is needed for to be sure that artifacts + # downloaded by the fetch steps are those expected by the + # project and that the possible patches are correctly applied. + # Moreover since we do not want any modification + # of this file (for reproducibility purpose), we prevent it by + # using --frozen flag (in CARGO_BUILD_FLAGS) and raise a clear error + # here is better than letting cargo tell (in case the file is missing) + # "Cargo.lock should be modified but --frozen was given" + + manifest_path = d.getVar("MANIFEST_PATH", True) + lockfile = os.path.join(os.path.dirname(manifest_path), "Cargo.lock") + if not os.path.exists(lockfile): + bb.fatal(f"{lockfile} file doesn't exist") + + # There are patched files and so Cargo.lock should be modified but we use + # --frozen so let's handle that modifications here. + # + # Note that a "better" (more elegant ?) would have been to use cargo update for + # patched packages: + # cargo update --offline -p package_1 -p package_2 + # But this is not possible since it requires that cargo local git db + # to be populated and this is not the case as we fetch git repo ourself. + + lockfile_orig = lockfile + ".orig" + if not os.path.exists(lockfile_orig): + shutil.copy(lockfile, lockfile_orig) + + newlines = [] + with open(lockfile_orig, "r") as f: + for line in f.readlines(): + if not line.startswith("source = \"git"): + newlines.append(line) + + with open(lockfile, "w") as f: + f.writelines(newlines) } do_configure[postfuncs] += "cargo_common_do_patch_paths"
Signature · 1 line
2.34.1
PATCH 3/3 drop rust-hello-world recipe
In reply to [email protected]From: Frederic Martinsons <[email protected]>
we now have more rust/cargo recipes and test that covers various use cases.
--- meta/conf/distro/include/maintainers.inc | 1 - meta/lib/oeqa/runtime/cases/rust.py | 12 ---------- .../packagegroup-core-tools-testapps.bb | 3 --- .../rust-hello-world/0001-enable-LTO.patch | 24 ------------------- .../rust-example/rust-hello-world_git.bb | 19 --------------- 5 files changed, 59 deletions(-) delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world_git.bb
meta/conf/distro/include/maintainers.inc +0 -1
@@ -734,7 +734,6 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <[email protected]>" RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <[email protected]>" RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <[email protected]>" RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>" -RECIPE_MAINTAINER:pn-rust-hello-world = "Randy MacLeod <[email protected]>" RECIPE_MAINTAINER:pn-rust-llvm = "Randy MacLeod <[email protected]>" RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-sato-screenshot = "Ross Burton <[email protected]>"
meta/lib/oeqa/runtime/cases/rust.py +0 -12
@@ -48,15 +48,3 @@ class RustCompileTest(OERuntimeTestCase): status, output = self.target.run('cargo run --manifest-path=/tmp/hello/Cargo.toml') msg = 'running compiled file failed, output: %s' % output self.assertEqual(status, 0, msg=msg) - -class RustHelloworldTest(OERuntimeTestCase): - @OETestDepends(['ssh.SSHTest.test_ssh']) - @OEHasPackage(['rust-hello-world']) - def test_rusthelloworld(self): - cmd = "rust-hello-world" - status, output = self.target.run(cmd) - msg = 'Exit status was not 0. Output: %s' % output - self.assertEqual(status, 0, msg=msg) - - msg = 'Incorrect output: %s' % output - self.assertEqual(output, "Hello, world!", msg=msg)
meta/recipes-core/packagegroups/packagegroup-core-tools-testapps.bb +0 -3
@@ -25,8 +25,6 @@ GOTOOLS ?= "go-helloworld" GOTOOLS:powerpc ?= "" GOTOOLS:riscv32 ?= "" -RUSTTOOLS ?= "rust-hello-world" - GSTEXAMPLES ?= "gst-examples" GSTEXAMPLES:riscv64 = ""
@@ -59,5 +57,4 @@ RDEPENDS:${PN} = "\ ${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', "${X11GLTOOLS}", "", d)} \ ${@bb.utils.contains('DISTRO_FEATURES', '3g', "${3GTOOLS}", "", d)} \ ${GOTOOLS} \ - ${RUSTTOOLS} \ "
meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch deleted +0 -24
@@ -1,24 +0,0 @@ -From fa40b874f6470ec11a8fd7b0c9909d0cdd2d6feb Mon Sep 17 00:00:00 2001 -From: Dan Callaghan <[email protected]> -Date: Fri, 5 Feb 2021 08:56:34 +1000 -Subject: [PATCH] enable LTO - -Upstream-Status: Pending ---- - Cargo.toml | 3 +++ - 1 file changed, 3 insertions(+) - -diff --git a/Cargo.toml b/Cargo.toml -index 7a2f6c8..cdb6b5d 100644 ---- a/Cargo.toml -+++ b/Cargo.toml -@@ -3,3 +3,6 @@ - name = "rust-hello-world" - version = "0.0.1" - authors = ["Cody P Schafer <[email protected]>"] -+ -+[profile.release] -+lto = true --- -2.28.0 -
meta/recipes-extended/rust-example/rust-hello-world_git.bb deleted +0 -19
@@ -1,19 +0,0 @@ -inherit cargo - -SRC_URI = "git://github.com/meta-rust/rust-hello-world.git;protocol=https;branch=master" -SRCREV="e0fa23f1a3cb1eb1407165bd2fc36d2f6e6ad728" -LIC_FILES_CHKSUM="file://COPYRIGHT;md5=e6b2207ac3740d2d01141c49208c2147" - -SRC_URI += "\ - file://0001-enable-LTO.patch \ - " - -UPSTREAM_CHECK_COMMITS = "1" - -SUMMARY = "Hello World by Cargo for Rust" -HOMEPAGE = "https://github.com/meta-rust/rust-hello-world" -LICENSE = "MIT | Apache-2.0" - -S = "${WORKDIR}/git" - -BBCLASSEXTEND = "native"
Signature · 1 line
2.34.1
Quoted 39 lines · Le mer. 2 août 2023, 15:16, <[email protected]> a écrit :
From: Frederic Martinsons <[email protected]>
This patch series force the usage of --frozen instead of --offline during cargo build.
This has the advantage to be sure that Cargo.lock file will not be modified.
Moreover, raise a clear error when Cargo.lock is not present. For this to work, we must modify ourself the Cargo.lock before building in case there are git repo that have been patched.The last commit was for removing rust-hello-world, patch to the documentation project will follow.
The following changes since commit
692e414aed5313ff275b69e93179aa7c559700f3:ffmpeg: Fix wrong code found with gas/2.41 (2023-08-01 09:49:14 +0100)
are available in the Git repository at:
https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
Frederic Martinsons (3):
cargo.bbclass: Use --frozen flag for cargo operations cargo_common.bbclass: Handle Cargo.lock modifications for git
dependencies
drop rust-hello-world recipemeta/classes-recipe/cargo.bbclass | 7 +++- meta/classes-recipe/cargo_common.bbclass | 41 +++++++++++++++++++ meta/conf/distro/include/maintainers.inc | 1 - meta/lib/oeqa/runtime/cases/rust.py | 12 ------ .../packagegroup-core-tools-testapps.bb | 3 -- .../rust-hello-world/0001-enable-LTO.patch | 24 ----------- .../rust-example/rust-hello-world_git.bb | 19 --------- 7 files changed, 47 insertions(+), 60 deletions(-) delete mode 100644 meta/recipes-extended/rust-example/rust-hello-world/0001-enable-LTO.patch delete mode 100644 meta/recipes-extended/rust-example/
Quoted 5 lines
rust-hello-world_git.bb
--
2.34.1
Sorry, I missed the v2 part in header. These are the follow up after remarks made yesterday (about removal of rust-hello-world and robustness when modifying Cargo.lock file in place)