cargo_common.bbclass: Fix typos
PATCH 1/2 cargo_common.bbclass: Fix typos
--- meta/classes-recipe/cargo_common.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/classes-recipe/cargo_common.bbclass +2 -2
@@ -56,7 +56,7 @@ cargo_common_do_configure () { [source.crates-io] replace-with = "bitbake" - local-registry = "/nonexistant" + local-registry = "/nonexistent" EOF fi
@@ -103,7 +103,7 @@ cargo_common_do_configure () { cat <<- EOF >> ${CARGO_HOME}/config [build] - # Use out of tree build destination to avoid poluting the source tree + # Use out of tree build destination to avoid polluting the source tree target-dir = "${B}/target" EOF fi
Signature · 1 line
2.35.1
PATCH 2/2 cargo_common.bbclass: Support local github repos
Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of do_compile, even if EXTRA_OECARGO_PATHS to add them as part of paths is used, as this is documented as only working for packages which exist in crates.io.
Add parsing of the SRC_URIs for git repos and include [patch] sections to redirect to the checked out source repos which the bitbake fetcher has already populated.
There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).
--- meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
meta/classes-recipe/cargo_common.bbclass +30 -0
@@ -116,6 +116,36 @@ cargo_common_do_configure () { EOF } +python cargo_common_do_patch_paths() { + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") + if not os.path.exists(cargo_config): + return + + src_uri = (d.getVar('SRC_URI') or "").split() + if len(src_uri) == 0: + return + + patches = dict() + workdir = d.getVar('WORKDIR') + fetcher = bb.fetch2.Fetch(src_uri, d) + for url in fetcher.urls: + ud = fetcher.ud[url] + if ud.type == 'git': + name = ud.parm.get('name') + destsuffix = ud.parm.get('destsuffix') + if name is not None and destsuffix is not None: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix)) + patches.setdefault(repo, []).append(path) + + with open(cargo_config, "a+") as config: + for k, v in patches.items(): + print('\n[patch."%s"]' % k, file=config) + for name in v: + print(name, file=config) +} +do_configure[postfuncs] += "cargo_common_do_patch_paths" + oe_cargo_fix_env () { export CC="${RUST_TARGET_CC}" export CXX="${RUST_TARGET_CXX}"
Signature · 1 line
2.35.1
Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos
There doesn’t seem to be a test case or an example for this. How can we ensure the code is correct?
Alex
Quoted 78 lines · Alex Kiernan
Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of
do_compile, even ifEXTRA_OECARGO_PATHSto add them as part ofpathsis used, as this is documented as only working for packages which exist in crates.io.Add parsing of the
SRC_URIsfor git repos and include[patch]sections to redirect to the checked out source repos which the bitbake fetcher has already populated.There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).
Signed-off-by Alex Kiernan <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++ 1 file changed, 30 insertions(+)meta/classes-recipe/cargo_common.bbclass +30
-0@@ -116,6 +116,36 @@ cargo_common_do_configure () { EOF } +python cargo_common_do_patch_paths() { + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") + if not os.path.exists(cargo_config): + return + + src_uri = (d.getVar('SRC_URI') or "").split() + if len(src_uri) == 0: + return + + patches = dict() + workdir = d.getVar('WORKDIR') + fetcher = bb.fetch2.Fetch(src_uri, d) + for url in fetcher.urls: + ud = fetcher.ud[url] + if ud.type == 'git': + name = ud.parm.get('name') + destsuffix = ud.parm.get('destsuffix') + if name is not None and destsuffix is not None: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix)) + patches.setdefault(repo, []).append(path) + + with open(cargo_config, "a+") as config: + for k, v in patches.items(): + print('\n[patch."%s"]' % k, file=config) + for name in v: + print(name, file=config) +} +do_configure[postfuncs] += "cargo_common_do_patch_paths" + oe_cargo_fix_env () { export CC="${RUST_TARGET_CC}"export CXX="${RUST_TARGET_CXX}"--
2.35.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#172284):
https://lists.openembedded.org/g/openembedded-core/message/172284
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos
I was wondering about how to do that - my test case was
https://github.com/akiernan/uuid-test, but that's clearly not adequate; the recipe there comes from cargo bitbake as bitbake -c update_crates doesn't deal with git dependencies, or dig down into transitive deps, which also needs solving.
I'm not clear if something like this could be added as part of the stuff that's in master-next for rust tests? Or if we need to build out something new.
Quoted 80 lines · Alexander Kanavin
There doesn’t seem to be a test case or an example for this. How can we ensure the code is correct?
Alex
Quoted 74 lines · Alex Kiernan
Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of
do_compile, even ifEXTRA_OECARGO_PATHSto add them as part ofpathsis used, as this is documented as only working for packages which exist in crates.io.Add parsing of the
SRC_URIsfor git repos and include[patch]sections to redirect to the checked out source repos which the bitbake fetcher has already populated.There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).
Signed-off-by Alex Kiernan <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++ 1 file changed, 30 insertions(+)meta/classes-recipe/cargo_common.bbclass +30
-1@@ -116,6 +116,36 @@ cargo_common_do_configure () { EOF } +python cargo_common_do_patch_paths() { + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") + if not os.path.exists(cargo_config): + return + + src_uri = (d.getVar('SRC_URI') or "").split() + if len(src_uri) == 0: + return + + patches = dict() + workdir = d.getVar('WORKDIR') + fetcher = bb.fetch2.Fetch(src_uri, d) + for url in fetcher.urls: + ud = fetcher.ud[url] + if ud.type == 'git': + name = ud.parm.get('name') + destsuffix = ud.parm.get('destsuffix') + if name is not None and destsuffix is not None: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix)) + patches.setdefault(repo, []).append(path) + + with open(cargo_config, "a+") as config: + for k, v in patches.items(): + print('\n[patch."%s"]' % k, file=config) + for name in v: + print(name, file=config) +} +do_configure[postfuncs] += "cargo_common_do_patch_paths" + oe_cargo_fix_env () { export CC="${RUST_TARGET_CC}" export CXX="${RUST_TARGET_CXX}" --2.35.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172284): https://lists.openembedded.org/g/openembedded-core/message/172284
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 1 line
Alex Kiernan
Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos
It would also help if there’s an actual recipe and component somewhere where the problem exists now. The uuid-test does look like a made up example.
Let’s merge the crate updater to core first, then we can think of handling other ways to specify dependencies. ‘cargo bitbake’ is pretty broken and dead.
Alex
Quoted 104 lines · Alex Kiernan
I was wondering about how to do that - my test case was
https://github.com/akiernan/uuid-test, but that's clearly not adequate; the recipe there comes fromcargo bitbakeasbitbake -c update_cratesdoesn't deal with git dependencies, or dig down into transitive deps, which also needs solving.I'm not clear if something like this could be added as part of the stuff that's in master-next for rust tests? Or if we need to build out something new.
Quoted 2 lines · Alexander Kanavin
There doesn’t seem to be a test case or an example for this. How can we
ensure the code is correct?
Quoted 4 lines
Alex
On Sun 30. Oct 2022 at 18.38, Alex Kiernan <[email protected]>
Quoted 7 lines · wrote:
Quoted 7 lines
Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of
do_compile, even ifEXTRA_OECARGO_PATHSto add them as part ofpathsis used, as this is documented as only working for packages which exist in crates.io.Quoted 8 lines
Quoted 8 lines
Add parsing of the
SRC_URIsfor git repos and include[patch]sections to redirect to the checked out source repos which the bitbake fetcher has already populated.There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934
).
Quoted 7 lines
Review
Quoted 7 lines
Signed-off-by Alex Kiernan <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++ 1 file changed, 30 insertions(+)(quoted diff) +0
-0b/meta/classes-recipe/cargo_common.bbclass
Quoted 19 lines
Quoted 19 lines
index
f503a001dd8e..63b13829084b100644meta/classes-recipe/cargo_common.bbclass +12
-0@@ -116,6 +116,36 @@ cargo_common_do_configure () { EOF } +python cargo_common_do_patch_paths() { + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") + if not os.path.exists(cargo_config): + return + + src_uri = (d.getVar('SRC_URI') or "").split() + if len(src_uri) == 0: + return + + patches = dict() + workdir = d.getVar('WORKDIR') + fetcher = bb.fetch2.Fetch(src_uri, d)Quoted 8 lines
meta/classes-recipe/cargo_common.bbclass cargo_common_do_configure () {
Quoted 8 lines
meta/classes-recipe/cargo_common.bbclass +8
-0+ for url in fetcher.urls: + ud = fetcher.ud[url] + if ud.type == 'git': + name = ud.parm.get('name') + destsuffix = ud.parm.get('destsuffix') + if name is not None and destsuffix is not None: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + path = '%s = { path = "%s" }' % (name,
os.path.join(workdir, destsuffix))Quoted 17 lines
Quoted 17 lines
meta/classes-recipe/cargo_common.bbclass +10
-1+ patches.setdefault(repo, []).append(path) + + with open(cargo_config, "a+") as config: + for k, v in patches.items(): + print('\n[patch."%s"]' % k, file=config) + for name in v: + print(name, file=config) +} +do_configure[postfuncs] += "cargo_common_do_patch_paths" + oe_cargo_fix_env () { export CC="${RUST_TARGET_CC}" export CXX="${RUST_TARGET_CXX}" --2.35.1
Quoted 3 lines
Quoted 3 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#172284):https://lists.openembedded.org/g/openembedded-core/message/172284
Quoted 3 lines
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
--
Alex Kiernan
Re: [OE-Core][PATCH 2/2] cargo_common.bbclass: Support local github repos
Quoted 3 lines · Alexander Kanavin
It would also help if there’s an actual recipe and component somewhere where the problem exists now. The uuid-test does look like a made up example.
That's fair... basically it was demonstrating the problem I have with a bunch of private repos, which I expect will crop up a lot in the embedded rust world - if you're doing public dev, everything exists as crates, in the private world you're consuming crates + internal stuff. Not sure where to try and find a real world example as I'd expect pretty much everyone will push to crates.io if their code is open - searching around for a bit, I've failed to find any (though a couple which have some dev only dependencies which come through git).
Quoted 2 lines
Let’s merge the crate updater to core first, then we can think of handling other ways to specify dependencies. ‘cargo bitbake’ is pretty broken and dead.
The crate updater feels like it's going to be directionally better - testing it locally, it works for things which only consume things from crates.io, but that's only one out of thirty or so components I have to deal with.
Quoted 100 lines
Alex
Quoted 97 lines · Alex Kiernan
I was wondering about how to do that - my test case was
https://github.com/akiernan/uuid-test, but that's clearly not adequate; the recipe there comes fromcargo bitbakeasbitbake -c update_cratesdoesn't deal with git dependencies, or dig down into transitive deps, which also needs solving.I'm not clear if something like this could be added as part of the stuff that's in master-next for rust tests? Or if we need to build out something new.
Quoted 72 lines · Alexander Kanavin
There doesn’t seem to be a test case or an example for this. How can we ensure the code is correct?
Alex
Quoted 66 lines · Alex Kiernan
Since disable network was added cargo configurations which reference git repos fail as they attempt to fetch across the network as part of
do_compile, even ifEXTRA_OECARGO_PATHSto add them as part ofpathsis used, as this is documented as only working for packages which exist in crates.io.Add parsing of the
SRC_URIsfor git repos and include[patch]sections to redirect to the checked out source repos which the bitbake fetcher has already populated.There are still cases which don't work - if you have multiple copies of the same repo with different revisions, there's currently no way to represent that and anything using a repo which has a virtual manifest will fail to build (see https://github.com/rust-lang/cargo/issues/4934).
Signed-off-by Alex Kiernan <[email protected]>--- meta/classes-recipe/cargo_common.bbclass | 30 ++++++++++++++++++++++++ 1 file changed, 30 insertions(+)meta/classes-recipe/cargo_common.bbclass +30
-1@@ -116,6 +116,36 @@ cargo_common_do_configure () { EOF } +python cargo_common_do_patch_paths() { + cargo_config = os.path.join(d.getVar("CARGO_HOME"), "config") + if not os.path.exists(cargo_config): + return + + src_uri = (d.getVar('SRC_URI') or "").split() + if len(src_uri) == 0: + return + + patches = dict() + workdir = d.getVar('WORKDIR') + fetcher = bb.fetch2.Fetch(src_uri, d) + for url in fetcher.urls: + ud = fetcher.ud[url] + if ud.type == 'git': + name = ud.parm.get('name') + destsuffix = ud.parm.get('destsuffix') + if name is not None and destsuffix is not None: + repo = '%s://%s%s' % (ud.proto, ud.host, ud.path) + path = '%s = { path = "%s" }' % (name, os.path.join(workdir, destsuffix)) + patches.setdefault(repo, []).append(path) + + with open(cargo_config, "a+") as config: + for k, v in patches.items(): + print('\n[patch."%s"]' % k, file=config) + for name in v: + print(name, file=config) +} +do_configure[postfuncs] += "cargo_common_do_patch_paths" + oe_cargo_fix_env () { export CC="${RUST_TARGET_CC}" export CXX="${RUST_TARGET_CXX}" --2.35.1
Quoted 8 lines
Quoted 8 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172284): https://lists.openembedded.org/g/openembedded-core/message/172284
Mute This Topic: https://lists.openembedded.org/mt/94668827/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=---
Alex Kiernan
--
Alex Kiernan