Use frozen flag for cargo build
cover Use frozen flag for cargo build
From: Frederic Martinsons <[email protected]>
This patch series force the usage of --frozen instead of --offline during cargo build. This has the advantage to be sure that Cargo.lock file will not be modified. Moreover, raise a clear error when Cargo.lock is not present.
For this to work, we must modify ourself the Cargo.lock before building in case there are git repo that have been patched.
The last commit is to handle the specific case of rust-hello-world recipe. If this recipe is no more necessary, we can drop this last patch.
The following changes since commit 9b5b850d6a6982bb8ff14dcfbb6769b293638293:
libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
are available in the Git repository at:
https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
Frederic Martinsons (3):
cargo.bbclass: Use --frozen flag for cargo operations cargo_common.bbclass: Handle Cargo.lock modifications for git
dependencies
cargo.bbclass: Offer a way to use --offline instead of --frozen
meta/classes-recipe/cargo.bbclass | 9 +++- meta/classes-recipe/cargo_common.bbclass | 43 +++++++++++++++++++ .../rust-example/rust-hello-world_git.bb | 2 + 3 files changed, 53 insertions(+), 1 deletion(-)
Signature · 1 line
2.34.1
PATCH 1/3 cargo.bbclass: Use --frozen flag for cargo operations
From: Frederic Martinsons <[email protected]>
It supersed the --offline flag and guarantee that Cargo.lock file will not be modified during the build.
--- meta/classes-recipe/cargo.bbclass | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
meta/classes-recipe/cargo.bbclass +6 -1
@@ -39,7 +39,12 @@ MANIFEST_PATH ??= "${S}/${CARGO_SRC_DIR}/Cargo.toml" RUSTFLAGS ??= "" BUILD_MODE = "${@['--release', ''][d.getVar('DEBUG_BUILD') == '1']}" -CARGO_BUILD_FLAGS = "-v --offline --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}" +# --frozen flag will prevent network access (which is required since only +# the do_fetch step is authorized to access network) +# and will require an up to date Cargo.lock file. +# This force the package being built to already ship a Cargo.lock, in the end +# this is what we want, at least, for reproducibility of the build. +CARGO_BUILD_FLAGS = "-v --frozen --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}" # This is based on the content of CARGO_BUILD_FLAGS and generally will need to # change if CARGO_BUILD_FLAGS changes.
Signature · 1 line
2.34.1
PATCH 2/3 cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies
In reply to [email protected]From: Frederic Martinsons <[email protected]>
Now we use --frozen, Cargo.lock cannot be modified by cargo build. These patched git dependencies requires that the git url is removed from Cargo.lock.
Fixes #15104
--- meta/classes-recipe/cargo_common.bbclass | 40 ++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
meta/classes-recipe/cargo_common.bbclass +40 -0
@@ -117,6 +117,8 @@ cargo_common_do_configure () { } python cargo_common_do_patch_paths() { + import shutil + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") if not os.path.exists(cargo_config): return
@@ -146,6 +148,44 @@ python cargo_common_do_patch_paths() { print('\n[patch."%s"]' % k, file=config) for name in v: print(name, file=config) + + if not patches: + return + + # Cargo.lock file is needed for to be sure that artifacts + # downloaded by the fetch steps are those expected by the + # project and that the possible patches are correctly applied. + # Moreover since we do not want any modification + # of this file (for reproducibility purpose), we prevent it by + # using --frozen flag (in CARGO_BUILD_FLAGS) and raise a clear error + # here is better than letting cargo tell (in case the file is missing) + # "Cargo.lock should be modified but --frozen was given" + + manifest_path = d.getVar("MANIFEST_PATH", True) + lockfile = os.path.join(os.path.dirname(manifest_path), "Cargo.lock") + if not os.path.exists(lockfile): + bb.fatal(f"{lockfile} file doesn't exist") + + # There are patched files and so Cargo.lock should be modified but we use + # --frozen so let's handle that modifications here. + # + # Note that a "better" (more elegant ?) would have been to use cargo update for + # patched packages: + # cargo update --offline -p package_1 -p package_2 + # But this is not possible since it requires that cargo local git db + # to be populated and this is not the case as we fetch git repo ourself. + + newlines = [] + with open(lockfile, "r") as f: + for line in f.readlines(): + if not line.startswith("source = \"git"): + newlines.append(line) + + lockfile_mod = lockfile + ".new" + with open(lockfile_mod, "w") as f: + f.writelines(newlines) + + shutil.move(lockfile_mod, lockfile) } do_configure[postfuncs] += "cargo_common_do_patch_paths"
Signature · 1 line
2.34.1
Re: [OE-core] [PATCH 2/3] cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies
Quoted 60 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
Now we use --frozen, Cargo.lock cannot be modified by cargo build. These patched git dependencies requires that the git url is removed from Cargo.lock.
Fixes #15104
Signed-off-by Frederic Martinsons <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 40 ++++++++++++++++++++++++ 1 file changed, 40 insertions(+)meta/classes-recipe/cargo_common.bbclass +32
-0@@ -117,6 +117,8 @@ cargo_common_do_configure () { } python cargo_common_do_patch_paths() { + import shutil + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") if not os.path.exists(cargo_config): return@@ -146,6 +148,44 @@ python cargo_common_do_patch_paths() { print('\n[patch."%s"]' % k, file=config) for name in v: print(name, file=config) + + if not patches: + return + + # Cargo.lock file is needed for to be sure that artifacts + # downloaded by the fetch steps are those expected by the + # project and that the possible patches are correctly applied. + # Moreover since we do not want any modification + # of this file (for reproducibility purpose), we prevent it by + # using --frozen flag (in CARGO_BUILD_FLAGS) and raise a clear error + # here is better than letting cargo tell (in case the file is missing) + # "Cargo.lock should be modified but --frozen was given" + + manifest_path = d.getVar("MANIFEST_PATH", True) + lockfile = os.path.join(os.path.dirname(manifest_path), "Cargo.lock") + if not os.path.exists(lockfile): + bb.fatal(f"{lockfile} file doesn't exist") + + # There are patched files and so Cargo.lock should be modified but we use + # --frozen so let's handle that modifications here. + # + # Note that a "better" (more elegant ?) would have been to use cargo update for + # patched packages: + # cargo update --offline -p package_1 -p package_2 + # But this is not possible since it requires that cargo local git db + # to be populated and this is not the case as we fetch git repo ourself. + + newlines = [] + with open(lockfile, "r") as f: + for line in f.readlines():
meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() { · line 190
Quoted 10 lines
meta/classes-recipe/cargo_common.bbclass +8
-0+ if not line.startswith("source = \"git"): + newlines.append(line) + + lockfile_mod = lockfile + ".new" + with open(lockfile_mod, "w") as f: + f.writelines(newlines) + + shutil.move(lockfile_mod, lockfile) } do_configure[postfuncs] += "cargo_common_do_patch_paths"
This patch isn't "wrong" but there are some tips we've picked up over the years of doing this which might help things in the future.
Some questions:
What happens if we run the cargo_common_do_patch_paths function multiple times?
What happens if the user hits Ctrl+C at an inopportune moment?
In this case the function is mostly ok but it might easily change in the future to something that isn't.
The best practise we've come to is the pattern of firstly copy X to X.orig if X.orig doesn't exist. This ensures you always start from a pristine backup. You can then directly open and write to X reading from X.orig.
This ensures that both of the questions I ask above become non-issues.
I'm know I'm being a little picky by mentioning this but I do want to raise awareness of the better pattern in general and make sure that examples in the class code are good as people will copy them!
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] cargo_common.bbclass: Handle Cargo.lock modifications for git dependencies
Thank you for the tips, will work on a v2 implementing that you suggested
Quoted 114 lines · Richard Purdie
Quoted 6 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
Now we use --frozen, Cargo.lock cannot be modified by cargo build. These patched git dependencies requires that the git url is removed from Cargo.lock.
Review
Quoted 8 lines
Fixes #15104
Signed-off-by Frederic Martinsons <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 40 ++++++++++++++++++++++++ 1 file changed, 40 insertions(+)(quoted diff) +0
-0b/meta/classes-recipe/cargo_common.bbclass
Quoted 17 lines
index
db54826ddb..01afb74640100644meta/classes-recipe/cargo_common.bbclass +3
-0@@ -117,6 +117,8 @@ cargo_common_do_configure () { } python cargo_common_do_patch_paths() { + import shutil + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") if not os.path.exists(cargo_config): return@@ -146,6 +148,44 @@ python cargo_common_do_patch_paths() { print('\n[patch."%s"]' % k, file=config) for name in v: print(name, file=config) +meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() { · line 161
Quoted 10 lines
meta/classes-recipe/cargo_common.bbclass +10
-0+ if not patches: + return + + # Cargo.lock file is needed for to be sure that artifacts + # downloaded by the fetch steps are those expected by the + # project and that the possible patches are correctly applied. + # Moreover since we do not want any modification + # of this file (for reproducibility purpose), we prevent it by + # using --frozen flag (in CARGO_BUILD_FLAGS) and raise a clear error + # here is better than letting cargo tell (in case the file ismissing)
meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() {
Quoted 4 lines
meta/classes-recipe/cargo_common.bbclass +4
-0+ # "Cargo.lock should be modified but --frozen was given" + + manifest_path = d.getVar("MANIFEST_PATH", True) + lockfile = os.path.join(os.path.dirname(manifest_path),"Cargo.lock")
meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() {
Quoted 4 lines
meta/classes-recipe/cargo_common.bbclass +4
-0+ if not os.path.exists(lockfile): + bb.fatal(f"{lockfile} file doesn't exist") + + # There are patched files and so Cargo.lock should be modified butwe use
meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() {
Quoted 3 lines
meta/classes-recipe/cargo_common.bbclass +3
-0+ # --frozen so let's handle that modifications here. + # + # Note that a "better" (more elegant ?) would have been to usecargo update for
meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() {
Quoted 4 lines
meta/classes-recipe/cargo_common.bbclass +4
-0+ # patched packages: + # cargo update --offline -p package_1 -p package_2 + # But this is not possible since it requires that cargo local git db + # to be populated and this is not the case as we fetch git repoourself.
Quoted 4 lines
meta/classes-recipe/cargo_common.bbclass +4
-0+ + newlines = [] + with open(lockfile, "r") as f: + for line in f.readlines():meta/classes-recipe/cargo_common.bbclass python cargo_common_do_patch_paths() { · line 190
Quoted 10 lines
meta/classes-recipe/cargo_common.bbclass +8
-0+ if not line.startswith("source = \"git"): + newlines.append(line) + + lockfile_mod = lockfile + ".new" + with open(lockfile_mod, "w") as f: + f.writelines(newlines) + + shutil.move(lockfile_mod, lockfile) } do_configure[postfuncs] += "cargo_common_do_patch_paths"This patch isn't "wrong" but there are some tips we've picked up over the years of doing this which might help things in the future.
Some questions:
What happens if we run the
cargo_common_do_patch_pathsfunction multiple times?
What happens if the user hits Ctrl+C at an inopportune moment?In this case the function is mostly ok but it might easily change in the future to something that isn't.
The best practise we've come to is the pattern of firstly copy X to X.orig if X.orig doesn't exist. This ensures you always start from a pristine backup. You can then directly open and write to X reading from X.orig.
This ensures that both of the questions I ask above become non-issues.
I'm know I'm being a little picky by mentioning this but I do want to raise awareness of the better pattern in general and make sure that examples in the class code are good as people will copy them!
Cheers,
Richard
PATCH 3/3 cargo.bbclass: Offer a way to use --offline instead of --frozen
In reply to [email protected]From: Frederic Martinsons <[email protected]>
And use that for rust-hello-world recipe that did not ship a Cargo.lock file.
--- meta/classes-recipe/cargo.bbclass | 4 +++- meta/classes-recipe/cargo_common.bbclass | 3 +++ meta/recipes-extended/rust-example/rust-hello-world_git.bb | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-)
meta/classes-recipe/cargo.bbclass +3 -1
@@ -39,12 +39,14 @@ MANIFEST_PATH ??= "${S}/${CARGO_SRC_DIR}/Cargo.toml" RUSTFLAGS ??= "" BUILD_MODE = "${@['--release', ''][d.getVar('DEBUG_BUILD') == '1']}" + # --frozen flag will prevent network access (which is required since only # the do_fetch step is authorized to access network) # and will require an up to date Cargo.lock file. # This force the package being built to already ship a Cargo.lock, in the end # this is what we want, at least, for reproducibility of the build. -CARGO_BUILD_FLAGS = "-v --frozen --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}" +CARGO_EXTRA_FLAGS = "${@['--frozen', '--offline'][d.getVar('CARGO_NO_FROZEN') == '1']}" +CARGO_BUILD_FLAGS = "-v ${CARGO_EXTRA_FLAGS} --target ${RUST_HOST_SYS} ${BUILD_MODE} --manifest-path=${MANIFEST_PATH}" # This is based on the content of CARGO_BUILD_FLAGS and generally will need to # change if CARGO_BUILD_FLAGS changes.
meta/classes-recipe/cargo_common.bbclass +3 -0
@@ -152,6 +152,9 @@ python cargo_common_do_patch_paths() { if not patches: return + if d.getVar("CARGO_NO_FROZEN") == 1: + return + # Cargo.lock file is needed for to be sure that artifacts # downloaded by the fetch steps are those expected by the # project and that the possible patches are correctly applied.
meta/recipes-extended/rust-example/rust-hello-world_git.bb +2 -0
@@ -14,6 +14,8 @@ SUMMARY = "Hello World by Cargo for Rust" HOMEPAGE = "https://github.com/meta-rust/rust-hello-world" LICENSE = "MIT | Apache-2.0" +CARGO_NO_FROZEN = "1" + S = "${WORKDIR}/git" BBCLASSEXTEND = "native"
Signature · 1 line
2.34.1
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Quoted 11 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
And use that for rust-hello-world recipe that did not ship a Cargo.lock file.
Signed-off-by Frederic Martinsons <[email protected]>--- meta/classes-recipe/cargo.bbclass | 4 +++- meta/classes-recipe/cargo_common.bbclass | 3 +++ meta/recipes-extended/rust-example/rust-hello-world_git.bb | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-)
Could you send a patch to the manual please adding the new variable CARGO_NO_FROZEN?
I'm torn on this one as we could have simpler code which did:
CARGO_FREEZE_OPTION = "--frozen"
and then have the hello-world recipe:
CARGO_FREEZE_OPTION = "--offline"
but I'm torn on whether that is better or not...
Cheers,
Richard
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Thanks Richard, but I'd prefer to drop this patch if possible. I think the rust-hello-world is no more useful and that we can replace it by a more "real" example like zvariant
from meta-selftest.
Let's see what others think about this
Quoted 30 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
And use that for rust-hello-world recipe that did not ship a Cargo.lock file.
Signed-off-by Frederic Martinsons <[email protected]>--- meta/classes-recipe/cargo.bbclass | 4 +++- meta/classes-recipe/cargo_common.bbclass | 3 +++ meta/recipes-extended/rust-example/rust-hello-world_git.bb | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-)Could you send a patch to the manual please adding the new variable
CARGO_NO_FROZEN?I'm torn on this one as we could have simpler code which did:
CARGO_FREEZE_OPTION= "--frozen"and then have the hello-world recipe:
CARGO_FREEZE_OPTION= "--offline"but I'm torn on whether that is better or not...
Cheers,
Richard
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Quoted 6 lines · Frédéric Martinsons
Thanks Richard, but I'd prefer to drop this patch if possible. I think the rust-hello-world is no more useful and that we can replace it by a more "real" example like zvariant from meta-selftest.
Let's see what others think about this
After some thought, I'm in favour. Lets drop it.
Cheers,
Richard
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Ok but this raises other questions then, this recipe is part of packagegroup-core-tools-tesapps
<https://git.openembedded.org/openembedded-core/tree/meta/recipes-core/packagegroups/packagegroup-core-tools-testapps.bb#n28>
and is part of a test case
<https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/runtime/cases/rust.py#n52>
So what to do with those ? Suppress them too or adapt them with another recipe (zvariant ?)
Quoted 14 lines · Richard Purdie
Quoted 6 lines · Frédéric Martinsons
Thanks Richard, but I'd prefer to drop this patch if possible. I think the rust-hello-world is no more useful and that we can replace it by a more "real" example like zvariant from meta-selftest.
Let's see what others think about this
After some thought, I'm in favour. Lets drop it.
Cheers,
Richard
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Quoted 2 lines · Frédéric Martinsons
Ok but this raises other questions then, this recipe is part of packagegroup-core-tools-tesapps
Just drop that.
Quoted 2 lines
and is part of a test case So what to do with those ? Suppress them too or adapt them with another recipe (zvariant ?)
I think the other test cases supersede this one so that can be dropped too?
Cheers,
Richard
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
I have no special feelings about rust-hello-world; if all places where it's currently used can be reworked to use real world components, that's totally fine.
Alex
Quoted 25 lines · Richard Purdie
Quoted 2 lines · Frédéric Martinsons
Ok but this raises other questions then, this recipe is part of packagegroup-core-tools-tesapps
Just drop that.
Quoted 2 lines
and is part of a test case So what to do with those ? Suppress them too or adapt them with another recipe (zvariant ?)
I think the other test cases supersede this one so that can be dropped too?
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#185222): https://lists.openembedded.org/g/openembedded-core/message/185222
Mute This Topic: https://lists.openembedded.org/mt/100458217/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
zariant rust recipe is tested for devtool cases there
<https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/devtool.py#n891>
The test case mentioned earlier seems to verify that we can run rust test code, do we want to keep testing
that rust binary can run ? If so, how to keep testing this while removing rust-hello-world recipes ?
maybe we can add a simple "println!("Hello world") " in this file
<https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/files/test.rs>
,
compile it like it is already done and run it
to check its output ?
What do you think?
Quoted 36 lines · Alexander Kanavin
I have no special feelings about rust-hello-world; if all places where it's currently used can be reworked to use real world components, that's totally fine.
Alex
Quoted 13 lines · Richard Purdie
Quoted 2 lines · Frédéric Martinsons
Ok but this raises other questions then, this recipe is part of packagegroup-core-tools-tesapps
Just drop that.
Quoted 2 lines
and is part of a test case So what to do with those ? Suppress them too or adapt them with another recipe (zvariant ?)
I think the other test cases supersede this one so that can be dropped too?
Quoted 7 lines
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#185222):https://lists.openembedded.org/g/openembedded-core/message/185222
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/100458217/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Quoted 7 lines · Frédéric Martinsons
zariant rust recipe is tested for devtool cases there
The test case mentioned earlier seems to verify that we can run rust test code, do we want to keep testing
that rust binary can run ? If so, how to keep testing this while removing rust-hello-world recipes ?
Quoted 5 lines
maybe we can add a simple "println!("Hello world") " in this file , compile it like it is already done and run it
to check its output ?What do you think?
Doesn't test_rust_compile already also run the rust code it compiled?
Cheers,
Richard
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Quoted 9 lines · Richard Purdie
Quoted 7 lines · Frédéric Martinsons
zariant rust recipe is tested for devtool cases there
The test case mentioned earlier seems to verify that we can run rust test code, do we want to keep testing
that rust binary can run ? If so, how to keep testing this while removing rust-hello-world recipes ?
Quoted 8 lines
Quoted 5 lines
maybe we can add a simple "println!("Hello world") " in this file , compile it like it is already done and run it
to check its output ?What do you think?
Doesn't
test_rust_compilealready also run the rust code it compiled?
Yes and checks the exit code - really not sure I see any additional value in turning it into another "Hello World". I'm fairly sure the cargo test gets you another "Hello World" test (but again only checks the return value). I think the rust-hello-world test is covered in other ways now.
Signature · 1 line
Alex Kiernan
Re: [OE-core] [PATCH 3/3] cargo.bbclass: Offer a way to use --offline instead of --frozen
Ok, thanks you all for the answers. I'll then add a new patch to make this cleansing and drop the patch I made to use --offline instead of --frozen (no more needed)
Quoted 30 lines · Alex Kiernan
Quoted 9 lines · Richard Purdie
Quoted 7 lines · Frédéric Martinsons
zariant rust recipe is tested for devtool cases there
The test case mentioned earlier seems to verify that we can run rust test code, do we want to keep testing
that rust binary can run ? If so, how to keep testing this while removing rust-hello-world recipes ?Quoted 8 lines
Quoted 5 lines
maybe we can add a simple "println!("Hello world") " in this file , compile it like it is already done and run it
to check its output ?What do you think?
Doesn't
test_rust_compilealready also run the rust code it compiled?Yes and checks the exit code - really not sure I see any additional value in turning it into another "Hello World". I'm fairly sure the cargo test gets you another "Hello World" test (but again only checks the return value). I think the rust-hello-world test is covered in other ways now.
--
Alex Kiernan
I think you need to better describe the benefits. What are the improvements? Can we just stay with --offline?
There's lots of intricate code added around cargo.lock handling, with special casing to revert to previous behavior via new variable, but I'm struggling to understand: to what end?
Alex
Quoted 43 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
This patch series force the usage of --frozen instead of --offline during cargo build. This has the advantage to be sure that Cargo.lock file will not be modified. Moreover, raise a clear error when Cargo.lock is not present.
For this to work, we must modify ourself the Cargo.lock before building in case there are git repo that have been patched.The last commit is to handle the specific case of rust-hello-world recipe. If this recipe is no more necessary, we can drop this last patch.
The following changes since commit
9b5b850d6a6982bb8ff14dcfbb6769b293638293:libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
are available in the Git repository at:
https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
Frederic Martinsons (3):
cargo.bbclass: Use --frozen flag for cargo operations cargo_common.bbclass: Handle Cargo.lock modifications for git
dependencies
cargo.bbclass: Offer a way to use --offline instead of --frozenmeta/classes-recipe/cargo.bbclass | 9 +++- meta/classes-recipe/cargo_common.bbclass | 43 +++++++++++++++++++ .../rust-example/rust-hello-world_git.bb | 2 + 3 files changed, 53 insertions(+), 1 deletion(-)--
2.34.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#185128): https://lists.openembedded.org/g/openembedded-core/message/185128
Mute This Topic: https://lists.openembedded.org/mt/100458213/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hello Alexander,
I tried to explain in the code and commit comments, you may also want to read the ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15104
and I try to explain and ask for help in a dedicated topic on this very
list: see https://lists.openembedded.org/g/openembedded-core/topic/100254129
But to summarize, Cargo.lock is required for various purposes and it includes the patching path process to work.
I think having no Cargo.lock for a rust recipe built under yocto is not desirable.
For the --frozen flag, it put harder constraints than --offline (--offline prevent network access but doesn't prevent modification
of Cargo.lock)
The special case to revert the behavior introduced was made to comply with rust-hello-world existence. I would like
to suppress this recipe instead of having this patch but I don't know if it is desirable and I'd like to think
for other points of views.
Quoted 66 lines · Alexander Kanavin
I think you need to better describe the benefits. What are the improvements? Can we just stay with --offline?
There's lots of intricate code added around cargo.lock handling, with special casing to revert to previous behavior via new variable, but I'm struggling to understand: to what end?
Alex
Quoted 4 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
This patch series force the usage of --frozen instead of --offline
during cargo build.
Quoted 1 line
This has the advantage to be sure that Cargo.lock file will not be
modified.
Quoted 5 lines
Moreover, raise a clear error when Cargo.lock is not present. For this to work, we must modify ourself the Cargo.lock before building in case there are git repo that have been patched.
The last commit is to handle the specific case of rust-hello-world
recipe.
Quoted 3 lines
If this recipe is no more necessary, we can drop this last patch.
The following changes since commit
9b5b850d6a6982bb8ff14dcfbb6769b293638293:Quoted 6 lines
libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
are available in the Git repository at:
15104-check-cargo-lock
Quoted 12 lines
Frederic Martinsons (3):
cargo.bbclass: Use --frozen flag for cargo operations cargo_common.bbclass: Handle Cargo.lock modifications for git
dependencies
cargo.bbclass: Offer a way to use --offline instead of --frozenmeta/classes-recipe/cargo.bbclass | 9 +++- meta/classes-recipe/cargo_common.bbclass | 43 +++++++++++++++++++ .../rust-example/rust-hello-world_git.bb | 2 + 3 files changed, 53 insertions(+), 1 deletion(-)Quoted 7 lines
--
2.34.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#185128):https://lists.openembedded.org/g/openembedded-core/message/185128
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/100458213/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Thanks for explaining, I suppose you can simply rework the patchset to remove rust-hello-world, and rework all places where it's used (you can grep the poky source tree :)
Alex
Quoted 77 lines · Frédéric Martinsons
Hello Alexander,
I tried to explain in the code and commit comments, you may also want to read the ticket: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15104
and I try to explain and ask for help in a dedicated topic on this very list: see https://lists.openembedded.org/g/openembedded-core/topic/100254129But to summarize, Cargo.lock is required for various purposes and it includes the patching path process to work. I think having no Cargo.lock for a rust recipe built under yocto is not desirable.
For the --frozen flag, it put harder constraints than --offline (--offline prevent network access but doesn't prevent modification of Cargo.lock)
The special case to revert the behavior introduced was made to comply with rust-hello-world existence. I would like to suppress this recipe instead of having this patch but I don't know if it is desirable and I'd like to think for other points of views.
Quoted 59 lines · Alexander Kanavin
I think you need to better describe the benefits. What are the improvements? Can we just stay with --offline?
There's lots of intricate code added around cargo.lock handling, with special casing to revert to previous behavior via new variable, but I'm struggling to understand: to what end?
Alex
Quoted 43 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
This patch series force the usage of --frozen instead of --offline during cargo build. This has the advantage to be sure that Cargo.lock file will not be modified. Moreover, raise a clear error when Cargo.lock is not present.
For this to work, we must modify ourself the Cargo.lock before building in case there are git repo that have been patched.The last commit is to handle the specific case of rust-hello-world recipe. If this recipe is no more necessary, we can drop this last patch.
The following changes since commit
9b5b850d6a6982bb8ff14dcfbb6769b293638293:libarchive: ignore CVE-2023-30571 (2023-07-30 12:00:13 +0100)
are available in the Git repository at:
https://gitlab.com/fmartinsons/openembedded-core 15104-check-cargo-lock
Frederic Martinsons (3):
cargo.bbclass: Use --frozen flag for cargo operations cargo_common.bbclass: Handle Cargo.lock modifications for git
dependencies
cargo.bbclass: Offer a way to use --offline instead of --frozenmeta/classes-recipe/cargo.bbclass | 9 +++- meta/classes-recipe/cargo_common.bbclass | 43 +++++++++++++++++++ .../rust-example/rust-hello-world_git.bb | 2 + 3 files changed, 53 insertions(+), 1 deletion(-)--
2.34.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#185128): https://lists.openembedded.org/g/openembedded-core/message/185128
Mute This Topic: https://lists.openembedded.org/mt/100458213/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-