cargo_common.bbclass: do not use buit-in git to fetch crates
PATCH 1/1 cargo_common.bbclass: do not use buit-in git to fetch crates
From: Frederic Martinsons <[email protected]>
When a project has a dependencies from a git repository and ssh access, this kind of thing in Cargo.toml for example:
[dependencies]
my-crate = { git = "ssh://git@smyserver:7999/myrepo.git, branch="main" }
do_compile step failed with authentication error:
| Updating git repository `ssh://git@myserver:7999/myrepo.git` | error: failed to get `my-crate` as a dependency of package `bscli v0.0.1 (/home/fmartinsons/bscli)` | | Caused by: | failed to load source for dependency `my-crate` | | Caused by: | Unable to update ssh://git@myserver:7999/myrepo.git?branch=main | | Caused by: | failed to fetch into: /home/fmartinsons/TAPOS_build/build-tapos/tmp/work/corei7-64-tapos-linux/bscli/v1.0.0+gitAUTOINC+5b5ba2aa3e-r0/cargo_home/git/db/cra-d8e163876777fda6 | | Caused by: | failed to authenticate when downloading repository | | * attempted ssh-agent authentication, but no usernames succeeded: `git` | | if the git CLI succeeds then `net.git-fetch-with-cli` may help here | https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli | | Caused by: | error authenticating: no auth sock variable; class=Ssh (23)
No problem if I run "cargo build" manually.
As the compiler suggested, it is corrected by adding git-fetch-with-cli in cargo config file.
--- meta/classes-recipe/cargo_common.bbclass | 6 ++++++ 1 file changed, 6 insertions(+)
meta/classes-recipe/cargo_common.bbclass +6 -0
@@ -114,6 +114,12 @@ cargo_common_do_configure () { progress.when = 'always' progress.width = 80 EOF + + cat <<- EOF >> ${CARGO_HOME}/config + + [net] + git-fetch-with-cli = true + EOF } oe_cargo_fix_env () {
Signature · 1 line
2.34.1
Not sure if I understand this fully, but network access in do_compile is not allowed. You need to place all crates into SRC_URI.
Alex
Quoted 73 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
When a project has a dependencies from a git repository and ssh access, this kind of thing in Cargo.toml for example:
[dependencies]
my-crate = { git = "ssh://git@smyserver:7999/myrepo.git, branch="main" }
do_compilestep failed with authentication error:| Updating git repository `ssh://git@myserver:7999/myrepo.git` | error: failed to get `my-crate` as a dependency of package `bscli v0.0.1 (/home/fmartinsons/bscli)` | | Caused by: | failed to load source for dependency `my-crate` | | Caused by: | Unable to update ssh://git@myserver:7999/myrepo.git?branch=main | | Caused by: | failed to fetch into: /home/fmartinsons/TAPOS_build/build-tapos/tmp/work/corei7-64-tapos-linux/bscli/v1.0.0+gitAUTOINC+5b5ba2aa3e-r0/cargo_home/git/db/cra-d8e163876777fda6 | | Caused by: | failed to authenticate when downloading repository | | * attempted ssh-agent authentication, but no usernames succeeded: `git` | | if the git CLI succeeds then `net.git-fetch-with-cli` may help here | https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli | | Caused by: | error authenticating: no auth sock variable; class=Ssh (23)No problem if I run "cargo build" manually.
As the compiler suggested, it is corrected by adding git-fetch-with-cli in cargo config file.
Signed-off-by Frederic Martinsons <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 6 ++++++ 1 file changed, 6 insertions(+)meta/classes-recipe/cargo_common.bbclass +6
-1@@ -114,6 +114,12 @@ cargo_common_do_configure () { progress.when = 'always' progress.width = 80 EOF + + cat <<- EOF >> ${CARGO_HOME}/config + + [net] + git-fetch-with-cli = true + EOF } oe_cargo_fix_env () { --2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178375): https://lists.openembedded.org/g/openembedded-core/message/178375
Mute This Topic: https://lists.openembedded.org/mt/97555635/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Yes I did do that in a first place (because this is what cargo bitbake give to me in a first place), but for git dependencies this doesn't work.
So , to take the example again I had that in SRC_URI (after correcting malformed url as tracked by
https://github.com/meta-rust/cargo-bitbake/issues/41):
"git://git@myserver:7999/myrepo.git;protocol=ssh;nobranch=1;name=my-crate;destsuffix=my-crate"
And also the SRCREV:SRCREV_FORMAT .= "_my-crate"
SRCREV_my-crate = "main"EXTRA_OECARGO_PATHS += "${WORKDIR}/my-crate"
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
Quoted 81 lines · Alexander Kanavin
Not sure if I understand this fully, but network access in
do_compileis not allowed. You need to place all crates intoSRC_URI.Alex
Quoted 73 lines · Frederic Martinsons
From: Frederic Martinsons <[email protected]>
When a project has a dependencies from a git repository and ssh access, this kind of thing in Cargo.toml for example:
[dependencies]
my-crate = { git = "ssh://git@smyserver:7999/myrepo.git, branch="main" }
do_compilestep failed with authentication error:| Updating git repository `ssh://git@myserver:7999/myrepo.git` | error: failed to get `my-crate` as a dependency of package `bscli v0.0.1 (/home/fmartinsons/bscli)` | | Caused by: | failed to load source for dependency `my-crate` | | Caused by: | Unable to update ssh://git@myserver:7999/myrepo.git?branch=main | | Caused by: | failed to fetch into: /home/fmartinsons/TAPOS_build/build-tapos/tmp/work/corei7-64-tapos-linux/bscli/v1.0.0+gitAUTOINC+5b5ba2aa3e-r0/cargo_home/git/db/cra-d8e163876777fda6 | | Caused by: | failed to authenticate when downloading repository | | * attempted ssh-agent authentication, but no usernames succeeded: `git` | | if the git CLI succeeds then `net.git-fetch-with-cli` may help here | https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli | | Caused by: | error authenticating: no auth sock variable; class=Ssh (23)No problem if I run "cargo build" manually.
As the compiler suggested, it is corrected by adding git-fetch-with-cli in cargo config file.
Signed-off-by Frederic Martinsons <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 6 ++++++ 1 file changed, 6 insertions(+)meta/classes-recipe/cargo_common.bbclass +6
-1@@ -114,6 +114,12 @@ cargo_common_do_configure () { progress.when = 'always' progress.width = 80 EOF + + cat <<- EOF >> ${CARGO_HOME}/config + + [net] + git-fetch-with-cli = true + EOF } oe_cargo_fix_env () { --2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178375): https://lists.openembedded.org/g/openembedded-core/message/178375
Mute This Topic: https://lists.openembedded.org/mt/97555635/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in SRC_URI in oe-core, so you could start by studying how they work.
Accessing network anywhere except do_fetch is not allowed for reproducibility and license compliance reasons (both break if that happens).
'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?
Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Try with this patch:
https://patchwork.yoctoproject.org/project/oe-core/patch/[email protected]/
Quoted 35 lines · Frederic Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 14 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).Quoted 5 lines
'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178417): https://lists.openembedded.org/g/openembedded-core/message/178417
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 1 line
Alex Kiernan
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
ISTR I tried this and ran into a different set of different problems.
Quoted 45 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 1 line
Alex Kiernan
Thanks for the patch alex, I did try it and the same happened (git clone failure during do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.toml
Quoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan
Are you listing your ssh repos in your SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.
Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan
Signature · 1 line
Alex Kiernan
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in do_compile is successful.
One major thing I forget to mention is that I use dunfell branch where I backport all the cargo/rust relatives from master.
The backport was functional and straightforward, I just have to keep the custom crate fetcher in openembedded-core (meta/lib/crate.py) because I'm stuck with bitbake 1.46.
-
Quoted 73 lines · Alex Kiernan
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan--
Alex Kiernan
You need to use kirkstone or newer to see the bitbake-driven network disabling in action. Can you try on current poky master? You don't need to port the whole build to that, just one sample recipe.
It is also expected that if you submit patches for master, they were tested on master.
Alex
Quoted 87 lines · Frédéric Martinsons
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in
do_compileis successful.One major thing I forget to mention is that I use dunfell branch where I backport all the cargo/rust relatives from master.
The backport was functional and straightforward, I just have to keep the custom crate fetcher in openembedded-core (meta/lib/crate.py) because I'm stuck with bitbake 1.46.-Quoted 73 lines · Alex Kiernan
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan--
Alex Kiernan
I understand, I'll pursue all my tests with the current poky master. Thanks
Quoted 99 lines · Alexander Kanavin
You need to use kirkstone or newer to see the bitbake-driven network disabling in action. Can you try on current poky master? You don't need to port the whole build to that, just one sample recipe.
It is also expected that if you submit patches for master, they were tested on master.
Alex
Quoted 87 lines · Frédéric Martinsons
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in
do_compileis successful.One major thing I forget to mention is that I use dunfell branch where I backport all the cargo/rust relatives from master.
The backport was functional and straightforward, I just have to keep the custom crate fetcher in openembedded-core (meta/lib/crate.py) because I'm stuck with bitbake 1.46.-Quoted 73 lines · Alex Kiernan
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan--
Alex Kiernan
Hello, I finally found why my setup didn't work. That was not related to https versus ssh.
For the patch provided in
https://patchwork.yoctoproject.org/project/oe-core/patch/[email protected]/
to work, the repository should have a Cargo.lock file. WIthout the Cargo.lock file, the cargo build will try to generate it (and so needs network access).
Note that I made a little change to the patch to support user in url (the patch process in cargo only work if url is exact match):
@@ -135,7 +135,10 @@ python cargo_common_do_patch_paths() {
name = ud.parm.get('name')
destsuffix = ud.parm.get('destsuffix') if name is not None and destsuffix is not None:
- repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
+ if ud.user:
+ repo = '%s://%s@%s%s' % (ud.proto, ud.user,
ud.host, ud.path)
+ else:
+ repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
path = '%s = { path = "%s" }' % (name,
os.path.join(workdir, destsuffix))
patches.setdefault(repo, []).append(path)Anyway, I would like to know how to you feel with this need for Cargo.lock problem ?
Because for it to be generated, the project needs to be built once and if this is expected for binary type to have Cargo.lock under version control:
https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries
It is not for "library" project.
Maybe we can add a "cargo generate-lockfile"
(https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html)
at the end of do_fetch ?
Moreover, I think we should add the --offline option to cargo build because the error generated will be more clear:
| DEBUG: Executing shell function do_compile
| NOTE: Using rust targets from /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/
| NOTE: cargo = /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargo | NOTE: cargo build -v --offline --target x86_64-poky-linux-gnu
--release --manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml | error: failed to get zbus-git-dep-test as a dependency of packagezbus-test v0.0.1 (/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)
| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | can't checkout from
'ssh://[email protected]/fmartinsons/zbus-git-dep-test.git': you are in the offline mode (--offline)
| WARNING: exit code 101 from a shell command.
Instead of
| DEBUG: Executing shell function do_compile
| NOTE: Using rust targets from /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/ | NOTE: cargo = /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargo | NOTE: cargo build -v --target x86_64-poky-linux-gnu --release --manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml
| Updating git repositoryssh://[email protected]/fmartinsons/zbus-git-dep-test.git
| warning: spurious network error (2 tries remaining): failed to
resolve address for gitlab.com: Temporary failure in name resolution;
class=Net (12)
| warning: spurious network error (1 tries remaining): failed to
resolve address for gitlab.com: Temporary failure in name resolution;
class=Net (12)
| error: failed to get zbus-git-dep-test as a dependency of packagezbus-test v0.0.1 (/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)
| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | failed to clone into: /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/cargo_home/git/db/zbus-git-dep-test-7f2b6322e3bb0cd5 | | Caused by: | network failure seems to have happened | if a proxy or similar is necessary `net.git-fetch-with-cli` may help here | https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli | | Caused by: | failed to resolve address for gitlab.com: Temporary failure in name
resolution; class=Net (12)|
| WARNING: exit code 101 from a shell command.
In any case, I'll be glad to work on this patch if you tell me what strategy to adopt.
Quoted 103 lines · Frédéric Martinsons
I understand, I'll pursue all my tests with the current poky master. Thanks
Quoted 99 lines · Alexander Kanavin
You need to use kirkstone or newer to see the bitbake-driven network disabling in action. Can you try on current poky master? You don't need to port the whole build to that, just one sample recipe.
It is also expected that if you submit patches for master, they were tested on master.
Alex
Quoted 87 lines · Frédéric Martinsons
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in
do_compileis successful.One major thing I forget to mention is that I use dunfell branch where I backport all the cargo/rust relatives from master.
The backport was functional and straightforward, I just have to keep the custom crate fetcher in openembedded-core (meta/lib/crate.py) because I'm stuck with bitbake 1.46.-Quoted 73 lines · Alex Kiernan
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan--
Alex Kiernan
--offline seems like the right thing to add, if it produces better errors.
Generating Cargo.lock on the other hand is not right. We rely on rust checking the source tree against the checksums in Cargo.lock (to prevent supply chain attacks), and this would completely subvert that. There could be better diagnostics around missing Cargo.lock, and steps to address the issue, but such steps must be manually taken (even though they could be codified in a special task, similar to how generating list of crates in SRC_URI is a task).
Alex
Quoted 215 lines · Frédéric Martinsons
Hello, I finally found why my setup didn't work. That was not related to https versus ssh.
For the patch provided in
https://patchwork.yoctoproject.org/project/oe-core/patch/[email protected]/
to work, the repository should have a Cargo.lock file. WIthout the Cargo.lock file, the cargo build will try to generate it (and so needs network access).Note that I made a little change to the patch to support user in url (the patch process in cargo only work if url is exact match):
(quoted diff) +4
-1@@ -135,7 +135,10 @@ python cargo_common_do_patch_paths() { name = ud.parm.get('name') destsuffix = ud.parm.get('destsuffix') if name is not None and destsuffix is not None: - repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + if ud.user: + repo = '%s://%s@%s%s' % (ud.proto, ud.user, ud.host, ud.path) + else: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix))patches.setdefault(repo, []).append(path)Anyway, I would like to know how to you feel with this need for Cargo.lock problem ?
Because for it to be generated, the project needs to be built once and if this is expected for binary type to have Cargo.lock under version control:
https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries
It is not for "library" project.Maybe we can add a "cargo generate-lockfile"
(https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html)
at the end ofdo_fetch?
Moreover, I think we should add the--offlineoption to cargo build because the error generated will be more clear:| DEBUG: Executing shell function
do_compile
| NOTE: Using rust targets from/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/
| NOTE: cargo =/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargo| NOTE: cargo build -v --offline --target x86_64-poky-linux-gnu
--release --manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml | error: failed to getzbus-git-dep-testas a dependency of packagezbus-test v0.0.1 (/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | can't checkout from'ssh://[email protected]/fmartinsons/zbus-git-dep-test.git': you are in the offline mode (--offline)
| WARNING: exit code 101 from a shell command.Instead of
| DEBUG: Executing shell function
do_compile
| NOTE: Using rust targets from/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/| NOTE: cargo =/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargo| NOTE: cargo build -v --target x86_64-poky-linux-gnu --release --manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml
| Updating git repositoryssh://[email protected]/fmartinsons/zbus-git-dep-test.git
| warning: spurious network error (2 tries remaining): failed to
resolve address for gitlab.com: Temporary failure in name resolution;
class=Net (12)
| warning: spurious network error (1 tries remaining): failed to
resolve address for gitlab.com: Temporary failure in name resolution;
class=Net (12)
| error: failed to getzbus-git-dep-testas a dependency of packagezbus-test v0.0.1 (/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | failed to clone into: /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/cargo_home/git/db/zbus-git-dep-test-7f2b6322e3bb0cd5 | | Caused by: | network failure seems to have happened | if a proxy or similar is necessary `net.git-fetch-with-cli` may help here | https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli | | Caused by: | failed to resolve address for gitlab.com: Temporary failure in nameresolution; class=Net (12)|
| WARNING: exit code 101 from a shell command.In any case, I'll be glad to work on this patch if you tell me what strategy to adopt.
Quoted 103 lines · Frédéric Martinsons
I understand, I'll pursue all my tests with the current poky master. Thanks
Quoted 99 lines · Alexander Kanavin
You need to use kirkstone or newer to see the bitbake-driven network disabling in action. Can you try on current poky master? You don't need to port the whole build to that, just one sample recipe.
It is also expected that if you submit patches for master, they were tested on master.
Alex
Quoted 87 lines · Frédéric Martinsons
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in
do_compileis successful.One major thing I forget to mention is that I use dunfell branch where I backport all the cargo/rust relatives from master.
The backport was functional and straightforward, I just have to keep the custom crate fetcher in openembedded-core (meta/lib/crate.py) because I'm stuck with bitbake 1.46.-Quoted 73 lines · Alex Kiernan
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan--
Alex Kiernan
Quoted 11 lines · Alexander Kanavin
--offline seems like the right thing to add, if it produces better errors.
Generating Cargo.lock on the other hand is not right. We rely on rust checking the source tree against the checksums in Cargo.lock (to prevent supply chain attacks), and this would completely subvert that. There could be better diagnostics around missing Cargo.lock, and steps to address the issue, but such steps must be manually taken (even though they could be codified in a special task, similar to how generating list of crates in
SRC_URIis a task).
Quoted 2 lines
Alex
Ok I see what you mean, generating the Cargo.lock in a dedicated task inside meta/classes-recipe/cargo-update-recipe-crates.bbclass or maybe a dedicated meta/classes-recipe/cargo-generate-lockfile.bbclass ?
Quoted 13 lines · Frédéric Martinsons
Quoted 11 lines · Alexander Kanavin
--offline seems like the right thing to add, if it produces better errors.
Generating Cargo.lock on the other hand is not right. We rely on rust checking the source tree against the checksums in Cargo.lock (to prevent supply chain attacks), and this would completely subvert that. There could be better diagnostics around missing Cargo.lock, and steps to address the issue, but such steps must be manually taken (even though they could be codified in a special task, similar to how generating list of crates in
SRC_URIis a task).
Quoted 6 lines
Quoted 2 lines
Alex
Ok I see what you mean, generating the Cargo.lock in a dedicated task inside meta/classes-recipe/cargo-update-recipe-crates.bbclass or maybe a dedicated meta/classes-recipe/cargo-generate-lockfile.bbclass ?
I'm really not sure we want Cargo.lock generation as part of the OE workflow - Cargo.lock should be part of your checked in sources for non-library projects.
--
Alex Kiernan
Understood, I searched how to add checksum support in crate fetcher but I don't know enough of bitbake to be effective.
I'll check in the Cargo.lock in my project, backport your patch and continue my exploration of the wonderful rust world on embedded device ^^
Thank you very much for the help on the subject guys and like I said earlier, if you need me to do some more tests to help the patch being merged in the master, I'm available.
Quoted 33 lines · Alex Kiernan
Quoted 2 lines · Frédéric Martinsons
On Tue, 14 Mar 2023 at 11:35, Alexander Kanavin <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
--offline seems like the right thing to add, if it produces better
errors.
Quoted 9 lines
Quoted 9 lines
Generating Cargo.lock on the other hand is not right. We rely on rust checking the source tree against the checksums in Cargo.lock (to prevent supply chain attacks), and this would completely subvert that. There could be better diagnostics around missing Cargo.lock, and steps to address the issue, but such steps must be manually taken (even though they could be codified in a special task, similar to how generating list of crates in
SRC_URIis a task).Quoted 6 lines
Quoted 2 lines
Alex
Ok I see what you mean, generating the Cargo.lock in a dedicated task inside meta/classes-recipe/cargo-update-recipe-crates.bbclass or maybe a dedicated meta/classes-recipe/cargo-generate-lockfile.bbclass
?
I'm really not sure we want Cargo.lock generation as part of the OE workflow - Cargo.lock should be part of your checked in sources for non-library projects.
--
Alex Kiernan
Quoted 1 line · Frédéric Martinsons
Understood, I searched how to add checksum support in crate fetcher but I don't know enough of bitbake to be effective.
You can check how for example http:// fetcher does it by utilizing verify_checksum() from lib/bb/fetch2/__init__.py in lib/bb/fetch2/wget.py.
Something like that should happen in crate.py as well.
Alex
A message that is not in the archive (sent off-list or not yet received): [email protected]
The other option is to add checksumming support to crate fetcher in bitbake. If all items in src_uri are verified by fetchers directly for integrity from checksums in the recipe, the cargo can generate cargo.lock anytime it wants to.
Alex
Quoted 274 lines · Alexander Kanavin via lists.openembedded.org
--offline seems like the right thing to add, if it produces better errors.
Generating Cargo.lock on the other hand is not right. We rely on rust checking the source tree against the checksums in Cargo.lock (to prevent supply chain attacks), and this would completely subvert that. There could be better diagnostics around missing Cargo.lock, and steps to address the issue, but such steps must be manually taken (even though they could be codified in a special task, similar to how generating list of crates in
SRC_URIis a task).Alex
Quoted 5 lines · Frédéric Martinsons
Hello, I finally found why my setup didn't work. That was not related to https versus ssh.
For the patch provided inhttps://patchwork.yoctoproject.org/project/oe-core/patch/[email protected]/
Quoted 21 lines
to work, the repository should have a Cargo.lock file. WIthout the Cargo.lock file, the cargo build will try to generate it (and so needs network access).
Note that I made a little change to the patch to support user in url (the patch process in cargo only work if url is exact match):
(quoted diff) +4
-1@@ -135,7 +135,10 @@ python cargo_common_do_patch_paths() { name = ud.parm.get('name') destsuffix = ud.parm.get('destsuffix') if name is not None and destsuffix is not None: - repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + if ud.user: + repo = '%s://%s@%s%s' % (ud.proto, ud.user, ud.host, ud.path) + else: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix))patches.setdefault(repo, []).append(path)Quoted 6 lines
Anyway, I would like to know how to you feel with this need for Cargo.lock problem ?
Because for it to be generated, the project needs to be built once and if this is expected for binary type to have Cargo.lock under version control:Quoted 11 lines
It is not for "library" project.
Maybe we can add a "cargo generate-lockfile"
(https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html)
at the end ofdo_fetch?
Moreover, I think we should add the--offlineoption to cargo build because the error generated will be more clear:| DEBUG: Executing shell function
do_compile
| NOTE: Using rust targets from
/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/Quoted 2 lines
| NOTE: cargo =
/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargoQuoted 2 lines
| NOTE: cargo build -v --offline --target x86_64-poky-linux-gnu --release
--manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml
Quoted 3 lines
| error: failed to get
zbus-git-dep-testas a dependency of package `zbus-test v0.0.1(/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)`
Quoted 10 lines
| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | can't checkout fromQuoted 9 lines
'ssh://[email protected]/fmartinsons/zbus-git-dep-test.git': you are in the offline mode (--offline)
| WARNING: exit code 101 from a shell command.Instead of
| DEBUG: Executing shell function
do_compile
| NOTE: Using rust targets from
/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/Quoted 2 lines
| NOTE: cargo =
/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargoQuoted 2 lines
| NOTE: cargo build -v --target x86_64-poky-linux-gnu --release
--manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml
Quoted 2 lines
| Updating git repository
ssh://[email protected]/fmartinsons/zbus-git-dep-test.git<http://[email protected]/fmartinsons/zbus-git-dep-test.git>
Quoted 9 lines
| warning: spurious network error (2 tries remaining): failed to resolve address for gitlab.com: Temporary failure in name resolution; class=Net (12)
| warning: spurious network error (1 tries remaining): failed to resolve address for gitlab.com: Temporary failure in name resolution; class=Net (12)
| error: failed to getzbus-git-dep-testas a dependency of package `zbus-test v0.0.1(/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)`
Quoted 11 lines
| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | failed to clone into:
/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/cargo_home/git/db/zbus-git-dep-test-7f2b6322e3bb0cd5Quoted 4 lines
| | Caused by: | network failure seems to have happened | if a proxy or similar is necessary `net.git-fetch-with-cli` may helphere
Quoted 1 line
|
https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli
Quoted 7 lines
| | Caused by: | failed to resolve address for gitlab.com: Temporary failure in nameresolution; class=Net (12)|
| WARNING: exit code 101 from a shell command.Quoted 7 lines
In any case, I'll be glad to work on this patch if you tell me what strategy to adopt.
Quoted 2 lines · Frédéric Martinsons
I understand, I'll pursue all my tests with the current poky master.
Thanks
Quoted 2 lines
Quoted 2 lines
On Mon, 13 Mar 2023 at 11:42, Alexander Kanavin <
Quoted 8 lines · [email protected]> wrote:
Quoted 8 lines
Quoted 8 lines
You need to use kirkstone or newer to see the bitbake-driven network disabling in action. Can you try on current poky master? You don't need to port the whole build to that, just one sample recipe.
It is also expected that if you submit patches for master, they were tested on master.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Alex
Quoted 4 lines · Frédéric Martinsons
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in
do_compileissuccessful.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
One major thing I forget to mention is that I use dunfell branch
where
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I backport all the cargo/rust relatives from master. The backport was functional and straightforward, I just have to
keep
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
the custom crate fetcher in openembedded-core (
meta/lib/crate.py) because I'm stuck with bitbake 1.46.-On Mon, 13 Mar 2023 at 10:46, Alex Kiernan <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Are you listing your ssh repos in your
SRC_URI? We run withnetwork
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
disabled everywhere other than fetch and with that patch we
don't see
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
fetches from local ssh repos in compile.
Quoted 2 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened
(git
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a wayof
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
supporting git dependencies in Cargo.toml
On Mon, 13 Mar 2023 at 10:23, Alex Kiernan <
Quoted 2 lines · [email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
ISTR I tried this and ran into a different set of different
problems.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in
yocto is still
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
evolving towards maturity. If you can see better ways to
do things,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
patches would be most welcome - but 'no network access
except in
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
bitbake fetchers' is a core rule.
Alex
Quoted 2 lines · Frédéric Martinsons
I understand there should not be a fetch during build
step , I'll try
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
to find why .
In the mean time, if this is a strong rule, shouldn't weinvoke "cargo
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
build" with --offline option ?
On Mon, 13 Mar 2023 at 09:38, Alexander Kanavin <
Quoted 4 lines · [email protected]> wrote:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line · Frédéric Martinsons
But the compile step tried to git clone anyway and
so it failed like I
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
showed in my first email.
You need to figure out why this happens, because it
should not. We
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
have plenty of recipes that list crates in
SRC_URIinoe-core, so you
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
could start by studying how they work.
Accessing network anywhere except
do_fetchis notallowed for
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
reproducibility and license compliance reasons (both
break if that
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
happens).
'cargo bitbake' is a 3rd party project with
questionable maintenance
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
status that we do not support. The official way is to
use
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
cargo-update-recipe-crates class.
Alex
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
--
Alex Kiernan--
Alex Kiernan-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#178480):
https://lists.openembedded.org/g/openembedded-core/message/178480
Mute This Topic: https://lists.openembedded.org/mt/97555635/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 5 lines · Alexander Kanavin
The other option is to add checksumming support to crate fetcher in bitbake. If all items in
src_uriare verified by fetchers directly for integrity from checksums in the recipe, the cargo can generate cargo.lock anytime it wants to.Alex
Ok, so If I understand correctly, that would mean a modification inside bitbake/lib/bb/fetch2/crate.py to add such support ?
And if we have that, where the Cargo.lock generation should be made ? it can only be during do_fetch step since it will
require network if a "git" dependencies is to be found in the project manifest.
Quoted 5 lines · Frédéric Martinsons
Quoted 3 lines · Alexander Kanavin
The other option is to add checksumming support to crate fetcher in bitbake. If all items in
src_uriare verified by fetchers directly for integrity from checksums in the recipe, the cargo can generate cargo.lock anytime it wants to.
Quoted 8 lines
Quoted 2 lines
Alex
Ok, so If I understand correctly, that would mean a modification inside
bitbake/lib/bb/fetch2/crate.pyto add such support ?
And if we have that, where the Cargo.lock generation should be made ? it can only be duringdo_fetchstep since it will
require network if a "git" dependencies is to be found in the project manifest.
You'd have to list all the crate checksums in your recipe, so the crate fetcher could check them.
--
Alex Kiernan
Quoted 24 lines · Frédéric Martinsons
Hello, I finally found why my setup didn't work. That was not related to https versus ssh.
For the patch provided in
https://patchwork.yoctoproject.org/project/oe-core/patch/[email protected]/
to work, the repository should have a Cargo.lock file. WIthout the Cargo.lock file, the cargo build will try to generate it (and so needs network access).Note that I made a little change to the patch to support user in url (the patch process in cargo only work if url is exact match):
(quoted diff) +4
-1@@ -135,7 +135,10 @@ python cargo_common_do_patch_paths() { name = ud.parm.get('name') destsuffix = ud.parm.get('destsuffix') if name is not None and destsuffix is not None: - repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + if ud.user: + repo = '%s://%s@%s%s' % (ud.proto, ud.user, ud.host, ud.path) + else: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix))
Quoted 2 lines
patches.setdefault(repo, []).append(path)
Cool, will pick that up... the code needs some tests which is where it stalled last time.
Quoted 8 lines
Anyway, I would like to know how to you feel with this need for Cargo.lock problem ?
Because for it to be generated, the project needs to be built once and if this is expected for binary type to have Cargo.lock under version control:
https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries
It is not for "library" project.
So I'm clear, you're building a library project here?
Could you consume your library into a wrapper project which brings Cargo.lock along with it? Other than running tests on it, I'm not sure what the value in building a library project is.
Quoted 6 lines
Maybe we can add a "cargo generate-lockfile"
(https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html)
at the end ofdo_fetch?
Moreover, I think we should add the--offlineoption to cargo build because the error generated will be more clear:
Let me chuck that across our code base, I was fairly sure I'd seen different problems with it, but it may have been whilst I was debugging this first time around and am mis-remembering.
Quoted 175 lines
| DEBUG: Executing shell function
do_compile
| NOTE: Using rust targets from/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/
| NOTE: cargo =/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargo| NOTE: cargo build -v --offline --target x86_64-poky-linux-gnu
--release --manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml | error: failed to getzbus-git-dep-testas a dependency of packagezbus-test v0.0.1 (/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | can't checkout from'ssh://[email protected]/fmartinsons/zbus-git-dep-test.git': you are in the offline mode (--offline)
| WARNING: exit code 101 from a shell command.Instead of
| DEBUG: Executing shell function
do_compile
| NOTE: Using rust targets from/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/rust-targets/| NOTE: cargo =/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/recipe-sysroot-native/usr/bin/cargo| NOTE: cargo build -v --target x86_64-poky-linux-gnu --release --manifest-path=/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git//Cargo.toml
| Updating git repositoryssh://[email protected]/fmartinsons/zbus-git-dep-test.git
| warning: spurious network error (2 tries remaining): failed to
resolve address for gitlab.com: Temporary failure in name resolution;
class=Net (12)
| warning: spurious network error (1 tries remaining): failed to
resolve address for gitlab.com: Temporary failure in name resolution;
class=Net (12)
| error: failed to getzbus-git-dep-testas a dependency of packagezbus-test v0.0.1 (/var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/git)| | Caused by: | failed to load source for dependency `zbus-git-dep-test` | | Caused by: | Unable to update ssh://[email protected]/fmartinsons/zbus-git-dep-test.git?branch=main | | Caused by: | failed to clone into: /var/lib/jenkins/YOCTO_POKY_MASTER_TEST/poky/build/tmp/work/core2-64-poky-linux/zbus-test/0.1.0.AUTOINC+507d3327e2-r0/cargo_home/git/db/zbus-git-dep-test-7f2b6322e3bb0cd5 | | Caused by: | network failure seems to have happened | if a proxy or similar is necessary `net.git-fetch-with-cli` may help here | https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli | | Caused by: | failed to resolve address for gitlab.com: Temporary failure in nameresolution; class=Net (12)|
| WARNING: exit code 101 from a shell command.In any case, I'll be glad to work on this patch if you tell me what strategy to adopt.
Quoted 103 lines · Frédéric Martinsons
I understand, I'll pursue all my tests with the current poky master. Thanks
Quoted 99 lines · Alexander Kanavin
You need to use kirkstone or newer to see the bitbake-driven network disabling in action. Can you try on current poky master? You don't need to port the whole build to that, just one sample recipe.
It is also expected that if you submit patches for master, they were tested on master.
Alex
Quoted 87 lines · Frédéric Martinsons
hmm, how is the network disabled ?
Because with the patch I provided , the git clone in
do_compileis successful.One major thing I forget to mention is that I use dunfell branch where I backport all the cargo/rust relatives from master.
The backport was functional and straightforward, I just have to keep the custom crate fetcher in openembedded-core (meta/lib/crate.py) because I'm stuck with bitbake 1.46.-Quoted 73 lines · Alex Kiernan
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 61 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 54 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 37 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178420): https://lists.openembedded.org/g/openembedded-core/message/178420
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan--
Alex Kiernan
Signature · 1 line
Alex Kiernan
Quoted 2 lines
Cool, will pick that up... the code needs some tests which is where it stalled last time.
Any help I can provide to you for having this patch merge ? Are you talking about unit testing ?
Quoted 10 lines
Quoted 3 lines
https://doc.rust-lang.org/cargo/faq.html#why-do-binaries-have-cargolock-in-version-control-but-not-libraries
It is not for "library" project.So I'm clear, you're building a library project here?
Could you consume your library into a wrapper project which brings Cargo.lock along with it? Other than running tests on it, I'm not sure what the value in building a library project is.
No, I build a binary but was not aware of the recommendation to add Cargo.lock in version control (and was not aware that was the missing of this file which make
cargo execute network operation during build),
I just started using rust several weeks ago so every day I learn new things ;)
Quoted 11 lines
Quoted 6 lines
Maybe we can add a "cargo generate-lockfile"
(https://doc.rust-lang.org/cargo/commands/cargo-generate-lockfile.html)
at the end ofdo_fetch?
Moreover, I think we should add the--offlineoption to cargo build because the error generated will be more clear:Let me chuck that across our code base, I was fairly sure I'd seen different problems with it, but it may have been whilst I was debugging this first time around and am mis-remembering.
Alright.
Quoted 4 lines · Alex Kiernan
Quoted 1 line · Frédéric Martinsons
Quoted 8 lines
Quoted 3 lines
Moreover, I think we should add the
--offlineoption to cargo build because the error generated will be more clear:Let me chuck that across our code base, I was fairly sure I'd seen different problems with it, but it may have been whilst I was debugging this first time around and am mis-remembering.
I'm wrong, seems to be fine.
Signature · 1 line
Alex Kiernan
I think I'll dig into the checksum capability on crate fetcher in the coming days.
For the offline option and the modication of the patch mentioned above, how do you want me to proceed?
Submit a dedicated patch for each of these (one for offline option and one for checksum if I can come up with a suitable patch)?
For the modification of the patch for using a local directory and not relying on git clone during build time, do you want me to provide something in particular?
Note that I saw the downloaded crates not being cleaned up from DL_DIR when I issue a bitbake -c cleanall but I need to confirm this on master branch.
Quoted 20 lines · Alex Kiernan
Quoted 4 lines · Alex Kiernan
Quoted 1 line · Frédéric Martinsons
Quoted 8 lines
Quoted 3 lines
Moreover, I think we should add the
--offlineoption to cargo build because the error generated will be more clear:Let me chuck that across our code base, I was fairly sure I'd seen different problems with it, but it may have been whilst I was debugging this first time around and am mis-remembering.
I'm wrong, seems to be fine.
--
Alex Kiernan
A message that is not in the archive (sent off-list or not yet received): [email protected]
For reference this was my test case:
https://github.com/akiernan/uuid-test
I was wondering if this was an ssh vs. https problem, but just checking we've a mix of ssh and https access to local git repos.
Quoted 76 lines · Alex Kiernan via lists.openembedded.org
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 55 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 48 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 6 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Quoted 33 lines
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
--
Alex Kiernan--
Alex Kiernan-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178424): https://lists.openembedded.org/g/openembedded-core/message/178424
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 1 line
Alex Kiernan
Thank you very much, I tested your project and it worked.
So I'll try https for my git instead of ssh.
Quoted 91 lines · Alex Kiernan
For reference this was my test case:
https://github.com/akiernan/uuid-test
I was wondering if this was an ssh vs. https problem, but just checking we've a mix of ssh and https access to local git repos.
Quoted 68 lines · Alex Kiernan via lists.openembedded.org
Are you listing your ssh repos in your
SRC_URI? We run with network disabled everywhere other than fetch and with that patch we don't see fetches from local ssh repos in compile.Quoted 55 lines · Frédéric Martinsons
Thanks for the patch alex, I did try it and the same happened (git clone failure during
do_compile), I'll
dig more on that and possibly submit patches if I found a way of supporting git dependencies in Cargo.tomlQuoted 48 lines · Alex Kiernan
ISTR I tried this and ran into a different set of different problems.
Quoted 6 lines · Alexander Kanavin
Note that we're all rust neophytes, and rust support in yocto is still evolving towards maturity. If you can see better ways to do things, patches would be most welcome - but 'no network access except in bitbake fetchers' is a core rule.
Quoted 33 lines
Alex
Quoted 26 lines · Frédéric Martinsons
I understand there should not be a fetch during build step , I'll try to find why .
In the mean time, if this is a strong rule, shouldn't we invoke "cargo build" with --offline option ?Quoted 19 lines · Alexander Kanavin
Quoted 2 lines · Frédéric Martinsons
But the compile step tried to git clone anyway and so it failed like I showed in my first email.
You need to figure out why this happens, because it should not. We have plenty of recipes that list crates in
SRC_URIin oe-core, so you could start by studying how they work.Accessing network anywhere except
do_fetchis not allowed for reproducibility and license compliance reasons (both break if that happens).'cargo bitbake' is a 3rd party project with questionable maintenance status that we do not support. The official way is to use cargo-update-recipe-crates class.
Alex
--
Alex Kiernan--
Alex KiernanQuoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178424): https://lists.openembedded.org/g/openembedded-core/message/178424
Mute This Topic: https://lists.openembedded.org/mt/97555635/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan