Skip to contentSkip to conversation

cargo_common.bbclass: Fix typos

PATCH In review Alex Kiernan 2 participants 6 messages Oct 30, 2022 – Oct 31, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 cargo_common.bbclass: Fix typos

---
 meta/classes-recipe/cargo_common.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes-recipe/cargo_common.bbclass +2 -2
index dea0fbe2f653..f503a001dd8e 100644
@@ -56,7 +56,7 @@ cargo_common_do_configure () {
 
 		[source.crates-io]
 		replace-with = "bitbake"
-		local-registry = "/nonexistant"
+		local-registry = "/nonexistent"
 		EOF
 	fi
 
@@ -103,7 +103,7 @@ cargo_common_do_configure () {
 		cat <<- EOF >> ${CARGO_HOME}/config
 
 		[build]
-		# Use out of tree build destination to avoid poluting the source tree
+		# Use out of tree build destination to avoid polluting the source tree
 		target-dir = "${B}/target"
 		EOF
 	fi
Signature · 1 line
2.35.1
#1 lore raw

PATCH 2/2 cargo_common.bbclass: Support local github repos

Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of do_compile, even if EXTRA_OECARGO_PATHS to add them as part of paths is used, as this is documented as only working for packages which exist in crates.io.

Add parsing of the SRC_URIs for git repos and include [patch] sections to redirect to the checked out source repos which the bitbake fetcher has already populated.

There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).

---
 meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
meta/classes-recipe/cargo_common.bbclass +30 -0
index f503a001dd8e..63b13829084b 100644
@@ -116,6 +116,36 @@ cargo_common_do_configure () {
 	EOF
 }
 
+python cargo_common_do_patch_paths() {
+    cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config")
+    if not os.path.exists(cargo_config):
+        return
+
+    src_uri = (d.getVar('SRC_URI') or "").split()
+    if len(src_uri) == 0:
+        return
+
+    patches = dict()
+    workdir = d.getVar('WORKDIR')
+    fetcher = bb.fetch2.Fetch(src_uri, d)
+    for url in fetcher.urls:
+        ud = fetcher.ud[url]
+        if ud.type == 'git':
+            name = ud.parm.get('name')
+            destsuffix = ud.parm.get('destsuffix')
+            if name is not None and destsuffix is not None:
+                repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
+                path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix))
+                patches.setdefault(repo, []).append(path)
+
+    with open(cargo_config, "a+") as config:
+        for k, v in patches.items():
+            print('\n[patch."%s"]' % k, file=config)
+            for name in v:
+                print(name, file=config)
+}
+do_configure[postfuncs] += "cargo_common_do_patch_paths"
+
 oe_cargo_fix_env () {
 	export CC="${RUST_TARGET_CC}"
 	export CXX="${RUST_TARGET_CXX}"
Signature · 1 line
2.35.1
#2 lore raw

Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos

There doesn’t seem to be a test case or an example for this. How can we ensure the code is correct?

Alex

Quoted 78 lines · Alex Kiernan

Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of do_compile, even if EXTRA_OECARGO_PATHS to add them as part of paths is used, as this is documented as only working for packages which exist in crates.io.

Add parsing of the SRC_URIs for git repos and include [patch] sections to redirect to the checked out source repos which the bitbake fetcher has already populated.

There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).

---
 meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
meta/classes-recipe/cargo_common.bbclass +30 -0
index f503a001dd8e..63b13829084b 100644
@@ -116,6 +116,36 @@ cargo_common_do_configure () {
        EOF
 }
 
+python cargo_common_do_patch_paths() {
+    cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config")
+    if not os.path.exists(cargo_config):
+        return
+
+    src_uri = (d.getVar('SRC_URI') or "").split()
+    if len(src_uri) == 0:
+        return
+
+    patches = dict()
+    workdir = d.getVar('WORKDIR')
+    fetcher = bb.fetch2.Fetch(src_uri, d)
+    for url in fetcher.urls:
+        ud = fetcher.ud[url]
+        if ud.type == 'git':
+            name = ud.parm.get('name')
+            destsuffix = ud.parm.get('destsuffix')
+            if name is not None and destsuffix is not None:
+                repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
+                path = '%s = { path = "%s" }' % (name,
 os.path.join(workdir, destsuffix))
+                patches.setdefault(repo, []).append(path)
+
+    with open(cargo_config, "a+") as config:
+        for k, v in patches.items():
+            print('\n[patch."%s"]' % k, file=config)
+            for name in v:
+                print(name, file=config)
+}
+do_configure[postfuncs] += "cargo_common_do_patch_paths"
+
 oe_cargo_fix_env () {
        export CC="${RUST_TARGET_CC}"
        export CXX="${RUST_TARGET_CXX}"

--
2.35.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#172284):
https://lists.openembedded.org/g/openembedded-core/message/172284
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos

I was wondering about how to do that - my test case was
https://github.com/akiernan/uuid-test, but that's clearly not adequate; the recipe there comes from cargo bitbake as bitbake -c update_crates doesn't deal with git dependencies, or dig down into transitive deps, which also needs solving.

I'm not clear if something like this could be added as part of the stuff that's in master-next for rust tests? Or if we need to build out something new.

Quoted 80 lines · Alexander Kanavin

There doesn’t seem to be a test case or an example for this. How can we ensure the code is correct?

Alex

Quoted 74 lines · Alex Kiernan

Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of do_compile, even if EXTRA_OECARGO_PATHS to add them as part of paths is used, as this is documented as only working for packages which exist in crates.io.

Add parsing of the SRC_URIs for git repos and include [patch] sections to redirect to the checked out source repos which the bitbake fetcher has already populated.

There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).

---
 meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
meta/classes-recipe/cargo_common.bbclass +30 -1
index f503a001dd8e..63b13829084b 100644
@@ -116,6 +116,36 @@ cargo_common_do_configure () {
        EOF
 }
 
+python cargo_common_do_patch_paths() {
+    cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config")
+    if not os.path.exists(cargo_config):
+        return
+
+    src_uri = (d.getVar('SRC_URI') or "").split()
+    if len(src_uri) == 0:
+        return
+
+    patches = dict()
+    workdir = d.getVar('WORKDIR')
+    fetcher = bb.fetch2.Fetch(src_uri, d)
+    for url in fetcher.urls:
+        ud = fetcher.ud[url]
+        if ud.type == 'git':
+            name = ud.parm.get('name')
+            destsuffix = ud.parm.get('destsuffix')
+            if name is not None and destsuffix is not None:
+                repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
+                path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix))
+                patches.setdefault(repo, []).append(path)
+
+    with open(cargo_config, "a+") as config:
+        for k, v in patches.items():
+            print('\n[patch."%s"]' % k, file=config)
+            for name in v:
+                print(name, file=config)
+}
+do_configure[postfuncs] += "cargo_common_do_patch_paths"
+
 oe_cargo_fix_env () {
        export CC="${RUST_TARGET_CC}"
        export CXX="${RUST_TARGET_CXX}"
--

2.35.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172284): https://lists.openembedded.org/g/openembedded-core/message/172284
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 1 line
Alex Kiernan
#4 lore raw

Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos

It would also help if there’s an actual recipe and component somewhere where the problem exists now. The uuid-test does look like a made up example.

Let’s merge the crate updater to core first, then we can think of handling other ways to specify dependencies. ‘cargo bitbake’ is pretty broken and dead.

Alex

Quoted 104 lines · Alex Kiernan

I was wondering about how to do that - my test case was
https://github.com/akiernan/uuid-test, but that's clearly not adequate; the recipe there comes from cargo bitbake as bitbake -c update_crates doesn't deal with git dependencies, or dig down into transitive deps, which also needs solving.

I'm not clear if something like this could be added as part of the stuff that's in master-next for rust tests? Or if we need to build out something new.

Quoted 2 lines · Alexander Kanavin

There doesn’t seem to be a test case or an example for this. How can we

ensure the code is correct?

Quoted 4 lines

Alex

On Sun 30. Oct 2022 at 18.38, Alex Kiernan <[email protected]>

Quoted 7 lines · wrote:
Quoted 7 lines

Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of do_compile, even if EXTRA_OECARGO_PATHS to add them as part of paths is used, as this is documented as only working for packages which exist in crates.io.

Quoted 8 lines
Quoted 8 lines

Add parsing of the SRC_URIs for git repos and include [patch] sections to redirect to the checked out source repos which the bitbake fetcher has already populated.

There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934

).

Quoted 7 lines

Review

Quoted 7 lines
---
 meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
(quoted diff) +0 -0

b/meta/classes-recipe/cargo_common.bbclass

Quoted 19 lines
Quoted 19 lines

index f503a001dd8e..63b13829084b 100644

meta/classes-recipe/cargo_common.bbclass +12 -0
@@ -116,6 +116,36 @@ cargo_common_do_configure () {
        EOF
 }
 
+python cargo_common_do_patch_paths() {
+    cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config")
+    if not os.path.exists(cargo_config):
+        return
+
+    src_uri = (d.getVar('SRC_URI') or "").split()
+    if len(src_uri) == 0:
+        return
+
+    patches = dict()
+    workdir = d.getVar('WORKDIR')
+    fetcher = bb.fetch2.Fetch(src_uri, d)
Quoted 8 lines

meta/classes-recipe/cargo_common.bbclass cargo_common_do_configure () {

Quoted 8 lines
meta/classes-recipe/cargo_common.bbclass +8 -0
+    for url in fetcher.urls:
+        ud = fetcher.ud[url]
+        if ud.type == 'git':
+            name = ud.parm.get('name')
+            destsuffix = ud.parm.get('destsuffix')
+            if name is not None and destsuffix is not None:
+                repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
+                path = '%s = { path = "%s" }' % (name,

os.path.join(workdir, destsuffix))

Quoted 17 lines
Quoted 17 lines
meta/classes-recipe/cargo_common.bbclass +10 -1
+                patches.setdefault(repo, []).append(path)
+
+    with open(cargo_config, "a+") as config:
+        for k, v in patches.items():
+            print('\n[patch."%s"]' % k, file=config)
+            for name in v:
+                print(name, file=config)
+}
+do_configure[postfuncs] += "cargo_common_do_patch_paths"
+
 oe_cargo_fix_env () {
        export CC="${RUST_TARGET_CC}"
        export CXX="${RUST_TARGET_CXX}"
--

2.35.1

Quoted 3 lines
Quoted 3 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#172284):

https://lists.openembedded.org/g/openembedded-core/message/172284

Quoted 3 lines
Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines
Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

--
Alex Kiernan

#5 lore raw

Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos

Quoted 3 lines · Alexander Kanavin

It would also help if there’s an actual recipe and component somewhere where the problem exists now. The uuid-test does look like a made up example.

That's fair... basically it was demonstrating the problem I have with a bunch of private repos, which I expect will crop up a lot in the embedded rust world - if you're doing public dev, everything exists as crates, in the private world you're consuming crates + internal stuff. Not sure where to try and find a real world example as I'd expect pretty much everyone will push to crates.io if their code is open - searching around for a bit, I've failed to find any (though a couple which have some dev only dependencies which come through git).

Quoted 2 lines

Let’s merge the crate updater to core first, then we can think of handling other ways to specify dependencies. ‘cargo bitbake’ is pretty broken and dead.

The crate updater feels like it's going to be directionally better - testing it locally, it works for things which only consume things from crates.io, but that's only one out of thirty or so components I have to deal with.

Quoted 100 lines

Alex

Quoted 97 lines · Alex Kiernan

I was wondering about how to do that - my test case was
https://github.com/akiernan/uuid-test, but that's clearly not adequate; the recipe there comes from cargo bitbake as bitbake -c update_crates doesn't deal with git dependencies, or dig down into transitive deps, which also needs solving.

I'm not clear if something like this could be added as part of the stuff that's in master-next for rust tests? Or if we need to build out something new.

Quoted 72 lines · Alexander Kanavin

There doesn’t seem to be a test case or an example for this. How can we ensure the code is correct?

Alex

Quoted 66 lines · Alex Kiernan

Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of do_compile, even if EXTRA_OECARGO_PATHS to add them as part of paths is used, as this is documented as only working for packages which exist in crates.io.

Add parsing of the SRC_URIs for git repos and include [patch] sections to redirect to the checked out source repos which the bitbake fetcher has already populated.

There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).

---
 meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
meta/classes-recipe/cargo_common.bbclass +30 -1
index f503a001dd8e..63b13829084b 100644
@@ -116,6 +116,36 @@ cargo_common_do_configure () {
        EOF
 }
 
+python cargo_common_do_patch_paths() {
+    cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config")
+    if not os.path.exists(cargo_config):
+        return
+
+    src_uri = (d.getVar('SRC_URI') or "").split()
+    if len(src_uri) == 0:
+        return
+
+    patches = dict()
+    workdir = d.getVar('WORKDIR')
+    fetcher = bb.fetch2.Fetch(src_uri, d)
+    for url in fetcher.urls:
+        ud = fetcher.ud[url]
+        if ud.type == 'git':
+            name = ud.parm.get('name')
+            destsuffix = ud.parm.get('destsuffix')
+            if name is not None and destsuffix is not None:
+                repo = '%s://%s%s' % (ud.proto, ud.host, ud.path)
+                path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix))
+                patches.setdefault(repo, []).append(path)
+
+    with open(cargo_config, "a+") as config:
+        for k, v in patches.items():
+            print('\n[patch."%s"]' % k, file=config)
+            for name in v:
+                print(name, file=config)
+}
+do_configure[postfuncs] += "cargo_common_do_patch_paths"
+
 oe_cargo_fix_env () {
        export CC="${RUST_TARGET_CC}"
        export CXX="${RUST_TARGET_CXX}"
--

2.35.1

Quoted 8 lines
Quoted 8 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172284): https://lists.openembedded.org/g/openembedded-core/message/172284
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

--
Alex Kiernan

--
Alex Kiernan