Skip to contentSkip to conversation

Introduction of cargo-c class and recipe

RFC PATCH Superseded [email protected] 2 participants 19 messages Aug 13, 2023 – Aug 15, 2023 2 messages not in the archive

v2 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v2 cover Introduction of cargo-c class and recipe

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin" IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass
TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

The following changes since commit 05095c116602d1a8c388cc02afffcc36230138f7:

qemuboot: Update hardcoded path to match new layout (2023-08-11 13:21:31 +0100)

are available in the Git repository at:

https://gitlab.com/fmartinsons/openembedded-core cargo-c-introduction

Frederic Martinsons (4):
rust: add cargo-c recipe
classes-recipe: add cargo_c.bbclass
rust: provide examples for C library generation in rust oeqa/runtime/rust: correct rust test

 .../rust/rust-c-lib-example-bin_git.bb        |   16 +
 .../rust/rust-c-lib-example-crates.inc        |   79 +
 .../rust/rust-c-lib-example_git.bb            |   15 +
 meta/classes-recipe/cargo_c.bbclass           |   41 +
 meta/conf/distro/include/maintainers.inc      |    2 +
 meta/lib/oeqa/runtime/cases/rust.py           |   16 +-
 meta/recipes-devtools/rust/cargo-c-crates.inc |  453 ++++
 meta/recipes-devtools/rust/cargo-c_0.9.18.bb  |   22 +
 .../rust/files/0001-Add-Cargo.lock-file.patch | 2180 +++++++++++++++++
 9 files changed, 2822 insertions(+), 2 deletions(-)
 create mode 100644 meta-selftest/recipes-devtools/rust/rust-c-lib-example-bin_git.bb
 create mode 100644 meta-selftest/recipes-devtools/rust/rust-c-lib-example-crates.inc
 create mode 100644 meta-selftest/recipes-devtools/rust/rust-c-lib-example_git.bb
 create mode 100644 meta/classes-recipe/cargo_c.bbclass
 create mode 100644 meta/recipes-devtools/rust/cargo-c-crates.inc
 create mode 100644 meta/recipes-devtools/rust/cargo-c_0.9.18.bb
 create mode 100644 meta/recipes-devtools/rust/files/0001-Add-Cargo.lock-file.patch
Signature · 1 line
2.34.1
#1 lore raw

PATCH v2 1/4 rust: add cargo-c recipe

From: Frederic Martinsons <[email protected]>

This package make it easier to generate C/C++ ABI compatible header, library and also generate package config file.

It is built around cbindgen (https://github.com/mozilla/cbindgen)
for the header generation, it compiles the library (static or dynamic) through cargo and finally handle the generation of a custom package config file.

---
 meta/conf/distro/include/maintainers.inc      |    1 +
 meta/recipes-devtools/rust/cargo-c-crates.inc |  453 ++++
 meta/recipes-devtools/rust/cargo-c_0.9.18.bb  |   22 +
 .../rust/files/0001-Add-Cargo.lock-file.patch | 2180 +++++++++++++++++
 4 files changed, 2656 insertions(+)
 create mode 100644 meta/recipes-devtools/rust/cargo-c-crates.inc
 create mode 100644 meta/recipes-devtools/rust/cargo-c_0.9.18.bb
 create mode 100644 meta/recipes-devtools/rust/files/0001-Add-Cargo.lock-file.patch
meta/conf/distro/include/maintainers.inc +1 -0
index 202e5739a0..d56b7c1e0e 100644
@@ -88,6 +88,7 @@ RECIPE_MAINTAINER:pn-bzip2 = "Denys Dmytriyenko <[email protected]>"
 RECIPE_MAINTAINER:pn-ca-certificates = "Alexander Kanavin <[email protected]>"
 RECIPE_MAINTAINER:pn-cairo = "Anuj Mittal <[email protected]>"
 RECIPE_MAINTAINER:pn-cargo = "Randy MacLeod <[email protected]>"
+RECIPE_MAINTAINER:pn-cargo-c = "Frederic Martinsons <[email protected]>"
 RECIPE_MAINTAINER:pn-cantarell-fonts = "Alexander Kanavin <[email protected]>"
 RECIPE_MAINTAINER:pn-ccache = "Robert Yang <[email protected]>"
 RECIPE_MAINTAINER:pn-cdrtools-native = "Yi Zhao <[email protected]>"
meta/recipes-devtools/rust/cargo-c-crates.inc new file +453 -0

453 lines not on this page · raw message

meta/recipes-devtools/rust/cargo-c_0.9.18.bb new file +22 -0
new file mode 100644
index 0000000000..1ff426aa9d
@@ -0,0 +1,22 @@
+SUMMARY ?= "cargo applet to build and install C-ABI compatible dynamic and static libraries."
+HOMEPAGE = "https://crates.io/crates/cargo-c"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = " \
+    file://LICENSE;md5=384ed0e2e0b2dac094e51fbf93fdcbe0 \
+"
+
+SRC_URI = " \
+	git://github.com/lu-zero/cargo-c.git;branch=master;protocol=https \
+	file://0001-Add-Cargo.lock-file.patch \
+"
+
+SRCREV = "4eaf39ebbbc9ab8f092adf487d5b53435511d619"
+S = "${WORKDIR}/git"
+
+inherit cargo cargo-update-recipe-crates pkgconfig
+
+DEPENDS = "openssl curl"
+
+require ${BPN}-crates.inc
+
+BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/rust/files/0001-Add-Cargo.lock-file.patch new file +2180 -0

2,180 lines not on this page · raw message

Signature · 1 line
2.34.1
#2 lore raw

PATCH v2 2/4 classes-recipe: add cargo_c.bbclass

In reply to [email protected]

From: Frederic Martinsons <[email protected]>

This class can be used inside rust recipe to generate a rust library that can be called by C/C++ code.

The rust recipe which uses this class has to only replace "inherit cargo" by "inherit cargo-c".

---
 meta/classes-recipe/cargo_c.bbclass | 41 +++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 meta/classes-recipe/cargo_c.bbclass
meta/classes-recipe/cargo_c.bbclass new file +41 -0
new file mode 100644
index 0000000000..c083f69c08
@@ -0,0 +1,41 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+##
+## Purpose:
+## This class is used by any recipes that want to compile a C ABI compatible
+## library with header and pkg config file
+
+inherit cargo pkgconfig
+
+# the binaries we will use
+CARGO_C_BUILD = "cargo-cbuild"
+CARGO_C_INSTALL = "cargo-cinstall"
+
+# We need cargo-c to compile for the target
+BASEDEPENDS:append = " cargo-c-native"
+
+do_compile[progress] = "outof:\s+(\d+)/(\d+)"
+cargo_c_do_compile() {
+    oe_cargo_fix_env
+    export RUSTFLAGS="${RUSTFLAGS}"
+    bbnote "Using rust targets from ${RUST_TARGET_PATH}"
+    bbnote "cargo-cbuild = $(which ${CARGO_C_BUILD})"
+    bbnote "${CARGO_C_BUILD} cbuild ${CARGO_BUILD_FLAGS}"
+    "${CARGO_C_BUILD}" cbuild ${CARGO_BUILD_FLAGS}
+}
+
+cargo_c_do_install() {
+    oe_cargo_fix_env
+    export RUSTFLAGS="${RUSTFLAGS}"
+    bbnote "cargo-cinstall = $(which ${CARGO_C_INSTALL})"
+    "${CARGO_C_INSTALL}" cinstall ${CARGO_BUILD_FLAGS} \
+        --destdir ${D} \
+        --prefix /usr \
+        --library-type cdylib
+}
+
+EXPORT_FUNCTIONS do_compile do_install
Signature · 1 line
2.34.1
#3 lore raw

PATCH v2 3/4 rust: provide examples for C library generation in rust

In reply to [email protected]

From: Frederic Martinsons <[email protected]>

rust-c-lib-example is a little rust code which provide a single function to print a formatted date (via the chrono crate) from an input timestamp in millisecond. It has the necessary FFI annotation and inherit cargo_c class for the C ABI compatible library generation.

rust-c-lib-example is meson project for the C code which will call the print_date function from rust-c-lib-example if no argument is provided, if any argument is provided it will print "Hello world in rust from C!"

add a runtime test case to check if all went well.

---
 .../rust/rust-c-lib-example-bin_git.bb        | 16 ++++
 .../rust/rust-c-lib-example-crates.inc        | 79 +++++++++++++++++++
 .../rust/rust-c-lib-example_git.bb            | 15 ++++
 meta/conf/distro/include/maintainers.inc      |  2 +
 meta/lib/oeqa/runtime/cases/rust.py           | 12 +++
 5 files changed, 124 insertions(+)
 create mode 100644 meta-selftest/recipes-devtools/rust/rust-c-lib-example-bin_git.bb
 create mode 100644 meta-selftest/recipes-devtools/rust/rust-c-lib-example-crates.inc
 create mode 100644 meta-selftest/recipes-devtools/rust/rust-c-lib-example_git.bb
meta-selftest/recipes-devtools/rust/rust-c-lib-example-bin_git.bb new file +16 -0
new file mode 100644
index 0000000000..47d878597a
@@ -0,0 +1,16 @@
+DESCRIPTION = "A simple example for C wrapper around a rust library"
+HOMEPAGE = "https://gitlab.com/fmartinsonsHome/rust-c-lib-example"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=cb9c441273ed8a029701a086befbfc63"
+
+SRC_URI = " \
+	git://gitlab.com/fmartinsonsHome/rust-c-lib-example.git;branch=main;protocol=https \
+"
+
+SRCREV = "fc53c457f69aa5221ec1f8619a007e8150db5e60"
+S = "${WORKDIR}/git"
+
+DEPENDS = "rust-c-lib-example"
+
+inherit meson pkgconfig
+
meta-selftest/recipes-devtools/rust/rust-c-lib-example-crates.inc new file +79 -0

79 lines not on this page · raw message

meta-selftest/recipes-devtools/rust/rust-c-lib-example_git.bb new file +15 -0
new file mode 100644
index 0000000000..6e7c250b92
@@ -0,0 +1,15 @@
+DESCRIPTION = "A simple example for rust C library"
+HOMEPAGE = "https://gitlab.com/fmartinsonsHome/rust-c-lib-example"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=cb9c441273ed8a029701a086befbfc63"
+
+SRC_URI = " \
+	git://gitlab.com/fmartinsonsHome/rust-c-lib-example.git;branch=main;protocol=https \
+"
+
+SRCREV = "fc53c457f69aa5221ec1f8619a007e8150db5e60"
+S = "${WORKDIR}/git"
+
+inherit cargo_c
+
+require ${BPN}-crates.inc
meta/conf/distro/include/maintainers.inc +2 -0
index 202e5739a0..b9fda0db09 100644
@@ -734,6 +734,8 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <[email protected]>"
 RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <[email protected]>"
 RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <[email protected]>"
+RECIPE_MAINTAINER:pn-rust-c-lib-example = "Frederic Martinsons <[email protected]>"
+RECIPE_MAINTAINER:pn-rust-c-lib-example-bin = "Frederic Martinsons <[email protected]>"
 RECIPE_MAINTAINER:pn-rust-llvm = "Randy MacLeod <[email protected]>"
 RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <[email protected]>"
 RECIPE_MAINTAINER:pn-sato-screenshot = "Ross Burton <[email protected]>"
meta/lib/oeqa/runtime/cases/rust.py +12 -0
index 9bf0312669..e709e4f8aa 100644
@@ -48,3 +48,15 @@ class RustCompileTest(OERuntimeTestCase):
         status, output = self.target.run('cargo run --manifest-path=/tmp/hello/Cargo.toml')
         msg = 'running compiled file failed, output: %s' % output
         self.assertEqual(status, 0, msg=msg)
+
+class RustCLibExampleTest(OERuntimeTestCase):
+    @OETestDepends(['ssh.SSHTest.test_ssh'])
+    @OEHasPackage(['rust-c-lib-example-bin'])
+    def test_rust_c_lib_example(self):
+        cmd = "rust-c-lib-example-bin test"
+        status, output = self.target.run(cmd)
+        msg = 'Exit status was not 0. Output: %s' % output
+        self.assertEqual(status, 0, msg=msg)
+
+        msg = 'Incorrect output: %s' % output
+        self.assertEqual(output, "Hello world in rust from C!", msg=msg)
Signature · 1 line
2.34.1
#4 lore raw

PATCH v2 4/4 oeqa/runtime/rust: correct rust test

In reply to [email protected]

From: Frederic Martinsons <[email protected]>

Since setUp of RustCompileTest use cls.tc.copyTo, those tests needs the scp command:

NOTE: ======================================================================
NOTE: FAIL: test_cargo_compile (rust.RustCompileTest)
NOTE: ----------------------------------------------------------------------
NOTE: Traceback (most recent call last):
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/case.py", line 53, in _oeSetUp
self.setUpMethod()
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/runtime/cases/rust.py", line 17, in setUp
cls.tc.target.copyTo(src, dst)
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 132, in copyTo
return self._run(scpCmd, ignore_status=False)
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 81, in _run
raise AssertionError("Command '%s' returned non-zero exit "
AssertionError: Command '['scp', '-o', 'ServerAliveCountMax=2', '-o', 'ServerAliveInterval=30', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '-r', '-P', '2222', '/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/files/test.rs', '[email protected]:/tmp/']' returned non-zero exit status 1:
sh: scp: not found
lost connection

---
 meta/lib/oeqa/runtime/cases/rust.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/lib/oeqa/runtime/cases/rust.py +2 -2
index e709e4f8aa..fa6fe486e4 100644
@@ -24,7 +24,7 @@ class RustCompileTest(OERuntimeTestCase):
         cls.tc.target.run('rm -r %s' % dirs)
 
     @OETestDepends(['ssh.SSHTest.test_ssh'])
-    @OEHasPackage(['rust'])
+    @OEHasPackage(['rust', 'openssh-scp'])
     def test_rust_compile(self):
         status, output = self.target.run('rustc /tmp/test.rs -o /tmp/test')
         msg = 'rust compile failed, output: %s' % output
@@ -35,7 +35,7 @@ class RustCompileTest(OERuntimeTestCase):
         self.assertEqual(status, 0, msg=msg)
 
     @OETestDepends(['ssh.SSHTest.test_ssh'])
-    @OEHasPackage(['cargo'])
+    @OEHasPackage(['cargo', 'openssh-scp'])
     def test_cargo_compile(self):
         status, output = self.target.run('cargo new /tmp/hello')
         msg = 'cargo new failed, output: %s' % output
Signature · 1 line
2.34.1
#5 lore raw
In reply to [email protected]
Quoted 11 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 7 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin" IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass
TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Cheers,

Richard

#6 lore raw
Quoted 23 lines · Richard Purdie
Quoted 6 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

Quoted 6 lines

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust

rust-c-lib-example-bin"

Quoted 6 lines

IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass
TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

Quoted 15 lines

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/rust/ cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/rust/ cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/rust/ cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Thanks Richard, will take care of that soon.

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

Quoted 4 lines

Cheers,

Richard

#7 lore raw
Quoted 40 lines · Frédéric Martinsons

Le dim. 13 août 2023, 12:23, Richard Purdie
<[email protected]> a écrit :

Quoted 23 lines · Frederic Martinsons
Quoted 11 lines

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 8 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin"
IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

Quoted 12 lines

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes- devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Quoted 4 lines

Thanks Richard, will take care of that soon.

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

It should just show up building the recipe (e.g. "bitbake cargo-c")?

There are further build failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/6361/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/6484/steps/14/logs/stdio

and the build isn't finished yet so Is suspect there will be more.

Cheers,

Richard

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 46 lines · Richard Purdie via lists.openembedded.org
Quoted 40 lines · Frédéric Martinsons

Le dim. 13 août 2023, 12:23, Richard Purdie
<[email protected]> a écrit :

Quoted 23 lines · Frederic Martinsons
Quoted 11 lines

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 8 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin"
IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

Quoted 12 lines

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes- devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Quoted 4 lines

Thanks Richard, will take care of that soon.

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

Quoted 8 lines

It should just show up building the recipe (e.g. "bitbake cargo-c")?

There are further build failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/6361/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/6484/steps/14/logs/stdio

and the build isn't finished yet so Is suspect there will be more.

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5606/steps/15/logs/stdio

not all of those failures are your patches but:

2023-08-13 12:38:57,894 - oe-selftest - INFO - ======================================================================
2023-08-13 12:38:57,894 - oe-selftest - INFO - FAIL: distrodata.Distrodata.test_maintainers (subunit.RemotedTestCase)
2023-08-13 12:38:57,894 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-08-13 12:38:57,894 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 115, in test_maintainers
    self.fail("""
  File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

AssertionError:
Unable to find recipes for the following entries in maintainers.inc:
rust-c-lib-example
rust-c-lib-example-bin

and this:

Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/runtime/cases/rust.py", line 31, in test_rust_compile
self.assertEqual(status, 0, msg=msg)
AssertionError: 127 != 0 : rust compile failed, output: sh: rustc: not found

probably is are, so 2 of the 6 failures. The other 4 are "mine" :/.

Cheers,

Richard

#10 lore raw
Quoted 77 lines · Richard Purdie
Quoted 41 lines · Richard Purdie via lists.openembedded.org
Quoted 40 lines · Frédéric Martinsons
Quoted 35 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 8 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin"
IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes- devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Quoted 10 lines
Quoted 4 lines

Thanks Richard, will take care of that soon.

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

It should just show up building the recipe (e.g. "bitbake cargo-c")?

There are further build failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/6361/steps/14/logs/stdio

Quoted 1 line

https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/6484/steps/14/logs/stdio

Quoted 2 lines

and the build isn't finished yet so Is suspect there will be more.

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5606/steps/15/logs/stdio

not all of those failures are your patches but:

2023-08-13 12:38:57,894 - oe-selftest - INFO -
======================================================================
2023-08-13 12:38:57,894 - oe-selftest - INFO - FAIL:
distrodata.Distrodata.test_maintainers (subunit.RemotedTestCase)
2023-08-13 12:38:57,894 - oe-selftest - INFO -
----------------------------------------------------------------------
2023-08-13 12:38:57,894 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call
last):
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 115, in test_maintainers

    self.fail("""
Quoted 9 lines

File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py", line 703, in fail
raise self.failureException(msg)
AssertionError:
Unable to find recipes for the following entries in maintainers.inc:
rust-c-lib-example
rust-c-lib-example-bin

I don't understand these, because rust-c-lib-example and rust-c-lib-example-bin have been added by PATCHV2 (3/4) in meta-selftest. ./meta-selftest/recipes-devtools/rust/rust-c-lib-example-bin_git.bb ./meta-selftest/recipes-devtools/rust/rust-c-lib-example_git.bb

Is this because they are in meta-selftest ? Should I remove them from maintainers.inc ?

Quoted 20 lines

and this:

Traceback (most recent call last):
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/runtime/cases/rust.py", line 31, in test_rust_compile
self.assertEqual(status, 0, msg=msg)
AssertionError: 127 != 0 : rust compile failed, output: sh: rustc: not
found

Quoted 1 line

I don't understand this one either, I simply add openssh-scp in require

package to run test_rust_compile since its setup function copy source on target via scp, rustc is provided by rust recipe and rust recipe is required to run this test. Any ideas ?

Quoted 7 lines

probably is are, so 2 of the 6 failures. The other 4 are "mine" :/.

Cheers,

Richard

#11 lore raw
Quoted 121 lines · Frédéric Martinsons
Quoted 77 lines · Richard Purdie
Quoted 41 lines · Richard Purdie via lists.openembedded.org
Quoted 40 lines · Frédéric Martinsons
Quoted 35 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 8 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin"
IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes- devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Quoted 10 lines
Quoted 4 lines

Thanks Richard, will take care of that soon.

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

It should just show up building the recipe (e.g. "bitbake cargo-c")?

There are further build failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/6361/steps/14/logs/stdio

Quoted 1 line

https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/6484/steps/14/logs/stdio

Quoted 2 lines

and the build isn't finished yet so Is suspect there will be more.

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5606/steps/15/logs/stdio

not all of those failures are your patches but:

2023-08-13 12:38:57,894 - oe-selftest - INFO -
======================================================================
2023-08-13 12:38:57,894 - oe-selftest - INFO - FAIL:
distrodata.Distrodata.test_maintainers (subunit.RemotedTestCase)
2023-08-13 12:38:57,894 - oe-selftest - INFO -
----------------------------------------------------------------------
2023-08-13 12:38:57,894 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call
last):
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 115, in test_maintainers

    self.fail("""
Quoted 9 lines

File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py", line 703, in fail
raise self.failureException(msg)
AssertionError:
Unable to find recipes for the following entries in maintainers.inc:
rust-c-lib-example
rust-c-lib-example-bin

I don't understand these, because rust-c-lib-example and rust-c-lib-example-bin have been added by PATCHV2 (3/4) in meta-selftest. ./meta-selftest/recipes-devtools/rust/rust-c-lib-example-bin_git.bb ./meta-selftest/recipes-devtools/rust/rust-c-lib-example_git.bb

Is this because they are in meta-selftest ? Should I remove them from maintainers.inc ?

Quoted 20 lines

and this:

Traceback (most recent call last):
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/runtime/cases/rust.py", line 31, in test_rust_compile
self.assertEqual(status, 0, msg=msg)
AssertionError: 127 != 0 : rust compile failed, output: sh: rustc: not
found

Quoted 5 lines
Quoted 1 line

I don't understand this one either, I simply add openssh-scp in require

package to run test_rust_compile since its setup function copy source on target via scp, rustc is provided by rust recipe and rust recipe is required to run this test. Any ideas ?

In fact, I do know where it's come from, I just had to look for the docstring of the decorator OEHasPackage since it tells that the writing @OEHasPackage(['rust', 'openssh-scp'])
means an "or" (test needs rust or openssh-scp), this is the wrong usage since the test needs both.

The V3 of the series I just sent should correct that, I also make cargo-c purely native but for the maintainers.inc issue, I have no clue...

Quoted 8 lines
Quoted 8 lines

probably is are, so 2 of the 6 failures. The other 4 are "mine" :/.

Cheers,

Richard

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw
Quoted 89 lines · Richard Purdie via lists.openembedded.org
Quoted 46 lines · Richard Purdie via lists.openembedded.org
Quoted 40 lines · Frédéric Martinsons

Le dim. 13 août 2023, 12:23, Richard Purdie
<[email protected]> a écrit :

Quoted 23 lines · Frederic Martinsons
Quoted 11 lines

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 8 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin"
IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

Quoted 12 lines

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes- devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Quoted 4 lines

Thanks Richard, will take care of that soon.

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

Quoted 8 lines

It should just show up building the recipe (e.g. "bitbake cargo-c")?

There are further build failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/6361/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/6484/steps/14/logs/stdio

and the build isn't finished yet so Is suspect there will be more.

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5606/steps/15/logs/stdio

not all of those failures are your patches but:

2023-08-13 12:38:57,894 - oe-selftest - INFO - ======================================================================
2023-08-13 12:38:57,894 - oe-selftest - INFO - FAIL: distrodata.Distrodata.test_maintainers (subunit.RemotedTestCase)
2023-08-13 12:38:57,894 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-08-13 12:38:57,894 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 115, in test_maintainers
    self.fail("""
  File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

AssertionError:
Unable to find recipes for the following entries in maintainers.inc:
rust-c-lib-example
rust-c-lib-example-bin

and this:

Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/runtime/cases/rust.py", line 31, in test_rust_compile
self.assertEqual(status, 0, msg=msg)
AssertionError: 127 != 0 : rust compile failed, output: sh: rustc: not found

Quoted 2 lines

probably is are, so 2 of the 6 failures. The other 4 are "mine" :/.

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

which leads to:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Cheers,

Richard

#14 lore raw
Quoted 116 lines · Richard Purdie
Quoted 45 lines · Richard Purdie via lists.openembedded.org
Quoted 43 lines · Richard Purdie via lists.openembedded.org
Quoted 42 lines · Frédéric Martinsons
Quoted 23 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

This is an RFC for introducing a new class and recipes that help building C-ABI compatible header and library around rust code.

The third patch add examples and test to demonstrate the usage and the good working of this use case.

Test have been passed with the following in local.conf:

Quoted 8 lines

CORE_IMAGE_EXTRA_INSTALL:append = " openssh-sshd openssh-scp cargo rust rust-c-lib-example-bin"
IMAGE_CLASSES += "testimage"
TEST_SUITES = "ping ssh rust"
# To use slirp option in testimage.bbclass TEST_RUNQEMUPARAMS = "slirp"
TEST_SERVER_IP = "127.0.0.1"
QEMU_USE_SLIRP = "1"

In testing it showed:

Quoted 12 lines

stdio: WARNING: cargo-c-0.9.18-r0 do_recipe_qa: QA Issue: Recipe cargo-c in /home/pokybuild/yocto-worker/a-full/build/meta/recipes- devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: cargo-c-native-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-cargo-c-0.9.18-r0 do_recipe_qa: QA Issue:
Recipe cargo-c in /home/pokybuild/yocto-worker/a- full/build/meta/recipes-devtools/rust/cargo-c_0.9.18.bb does not contain a SUMMARY. Please add an entry. [missing-metadata]

Thanks Richard, will take care of that soon.

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

Can you tell me what commands do you run to have that sanity check? It will avoid me to do the same mistake next time I'll add a recipe.

It should just show up building the recipe (e.g. "bitbake cargo-c")?

There are further build failures:

https://autobuilder.yoctoproject.org/typhoon/#/builders/101/builds/6361/steps/14/logs/stdio

Quoted 1 line
Quoted 1 line

https://autobuilder.yoctoproject.org/typhoon/#/builders/110/builds/6484/steps/14/logs/stdio

Quoted 4 lines
Quoted 2 lines

and the build isn't finished yet so Is suspect there will be more.

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5606/steps/15/logs/stdio

Quoted 4 lines

not all of those failures are your patches but:

2023-08-13 12:38:57,894 - oe-selftest - INFO -

======================================================================

Quoted 1 line
2023-08-13 12:38:57,894 - oe-selftest - INFO - FAIL:

distrodata.Distrodata.test_maintainers (subunit.RemotedTestCase)

Quoted 1 line
2023-08-13 12:38:57,894 - oe-selftest - INFO -

----------------------------------------------------------------------

Quoted 1 line
2023-08-13 12:38:57,894 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call last):

Quoted 1 line

File

"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/distrodata.py", line 115, in test_maintainers

Quoted 2 lines
    self.fail("""

File

"/home/pokybuild/yocto-worker/oe-selftest-centos/build/buildtools/sysroots/x86_64-pokysdk-linux/usr/lib/python3.11/unittest/case.py", line 703, in fail

Quoted 11 lines

raise self.failureException(msg)
AssertionError:
Unable to find recipes for the following entries in maintainers.inc:
rust-c-lib-example
rust-c-lib-example-bin

and this:

Traceback (most recent call last):
File

"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f

Quoted 3 lines

return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File

"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f

Quoted 3 lines

return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File

"/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/runtime/cases/rust.py", line 31, in test_rust_compile

Quoted 2 lines

self.assertEqual(status, 0, msg=msg)
AssertionError: 127 != 0 : rust compile failed, output: sh: rustc: not

found

Quoted 4 lines

probably is are, so 2 of the 6 failures. The other 4 are "mine" :/.

and a reproducibility failure:

Quoted 6 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

which leads to:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Argh, this makes me remember of
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

Quoted 7 lines

Cheers,

Richard

#15 lore raw
Quoted 9 lines · Frédéric Martinsons
Quoted 7 lines · Richard Purdie

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

which leads to:

Quoted 8 lines
Quoted 2 lines

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Argh, this makes me remember
of https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

At some point we're going to have to dive in and fix the reproducibility issues so I'm reluctant to take more patches with that set...

Cheers,

Richard

#16 lore raw
Quoted 16 lines · Richard Purdie
Quoted 6 lines · Frédéric Martinsons
Quoted 4 lines · Richard Purdie

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

Quoted 4 lines
Quoted 4 lines

which leads to:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Quoted 3 lines
Quoted 1 line
Quoted 8 lines
Quoted 4 lines

Argh, this makes me remember
of https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

At some point we're going to have to dive in and fix the reproducibility issues so I'm reluctant to take more patches with that set...

I understand, in the meantime, I issue a simple bibtake cargo-c and see these (along with the missing SUMMARY):

WARNING: cargo-c-0.9.18-r0 do_package_qa: QA Issue: File
/usr/bin/.debug/cargo-cinstall in package cargo-c-dbg contains reference to TMPDIR
File /usr/bin/.debug/cargo-cbuild in package cargo-c-dbg contains reference to TMPDIR
File /usr/bin/.debug/cargo-ctest in package cargo-c-dbg contains reference to TMPDIR
File /usr/bin/.debug/cargo-capi in package cargo-c-dbg contains reference to TMPDIR [buildpaths]

I guess this doesn't help to be reproducible, I'll look at those soon.

Sadly, I didn't see any of these warnings during my tests, I just have to look sharply for the next time.

Quoted 2 lines

Cheers,

Quoted 5 lines

Richard

#17 lore raw
In reply to Richard Purdie
Quoted 16 lines · Richard Purdie
Quoted 6 lines · Frédéric Martinsons
Quoted 4 lines · Richard Purdie

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

Quoted 4 lines
Quoted 4 lines

which leads to:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Quoted 3 lines
Quoted 1 line
Quoted 8 lines
Quoted 4 lines

Argh, this makes me remember
of https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

At some point we're going to have to dive in and fix the reproducibility issues so I'm reluctant to take more patches with that set...

It will hardly be doable to correct the reproducibility of rust packages since rust/cargo
itself have issues upstream. I take a look and there are lot of open issues cargo:
https://github.com/rust-lang/cargo/issues?q=is%3Aissue+is%3Aopen+label%3AA-reproducibility+
rust:
https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+label%3AA-reproducibility

There was an RFC accepted back in may (
https://github.com/rust-lang/rfcs/pull/3127) and implementation progress have a tracking issue in cargo (
https://github.com/rust-lang/cargo/issues/12137) and in rust
(https://github.com/rust-lang/rust/issues/111540).

Don't know if it will fix our root issue (
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14875) but I don't see how to workaround all of that without this upstream work.

For cargo-c specific issues, I'll try to make things better (the TMPDIR in debug symbols) but I'm wondering
something, reproducibility is desirable for all generated artifacts or is it only for target ones ? I mean,
the cargo-c is used in native so I'm wondering if making it native only would be acceptable ?

Quoted 2 lines

Cheers,

Quoted 5 lines

Richard

#18 lore raw
Quoted 30 lines · Frédéric Martinsons
Quoted 19 lines · Richard Purdie
Quoted 9 lines · Frédéric Martinsons
Quoted 7 lines · Richard Purdie

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

which leads to:

Quoted 8 lines
Quoted 2 lines

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Argh, this makes me remember
of https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

Quoted 5 lines

At some point we're going to have to dive in and fix the reproducibility issues so I'm reluctant to take more patches with that
set...

Quoted 10 lines

It will hardly be doable to correct the reproducibility of rust packages since rust/cargo
itself have issues upstream. I take a look and there are lot of open issues
cargo:
https://github.com/rust-lang/cargo/issues?q=is%3Aissue+is%3Aopen+labe
l%3AA-reproducibility+
rust:
https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+label
%3AA-reproducibility

I believe there is one problematic reference left which causes problems for core rust. Fixing that issue would be the best place to start and then we can work from there.

We do already have a lot of good reproducibility pieces in the toolchain so not every upstream bug will apply to us.

Quoted 10 lines

There was an RFC accepted back in may
(https://github.com/rust-lang/rfcs/pull/3127) and implementation progress have a tracking issue in cargo
(https://github.com/rust-lang/cargo/issues/12137) and in rust
(https://github.com/rust-lang/rust/issues/111540).

Don't know if it will fix our root issue
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=14875) but I don't see how to workaround all of that without this upstream work.

Quoted 6 lines

For cargo-c specific issues, I'll try to make things better (the TMPDIR in debug symbols) but I'm wondering
something, reproducibility is  desirable for all generated artifacts or is it only for target ones ?  I mean,
the cargo-c is used in native so I'm wondering if making it native only would be acceptable ?

That would certainly avoid this issue. It would also make on target testing of the functionality not possible.

Cheers,

Richard

#19 lore raw
Quoted 44 lines · Richard Purdie
Quoted 11 lines · Frédéric Martinsons
Quoted 7 lines · Richard Purdie
Quoted 6 lines · Frédéric Martinsons
Quoted 4 lines · Richard Purdie

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

which leads to:

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Quoted 15 lines
Quoted 8 lines
Quoted 7 lines
Quoted 1 line

Argh, this makes me remember
of https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

Quoted 5 lines

At some point we're going to have to dive in and fix the reproducibility issues so I'm reluctant to take more patches with that
set...

Quoted 10 lines

It will hardly be doable to correct the reproducibility of rust packages since rust/cargo
itself have issues upstream. I take a look and there are lot of open issues
cargo:
https://github.com/rust-lang/cargo/issues?q=is%3Aissue+is%3Aopen+labe
l%3AA-reproducibility+
rust:
https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+label
%3AA-reproducibility

Quoted 7 lines

I believe there is one problematic reference left which causes problems for core rust. Fixing that issue would be the best place to start and then we can work from there.

We do already have a lot of good reproducibility pieces in the toolchain so not every upstream bug will apply to us.

I'm afraid I do not see what is your point, can you elaborate please? What is the "problematic reference left which causes problems
for core rust" you refer to?

Quoted 17 lines
Quoted 10 lines

There was an RFC accepted back in may
(https://github.com/rust-lang/rfcs/pull/3127) and implementation progress have a tracking issue in cargo
(https://github.com/rust-lang/cargo/issues/12137) and in rust
(https://github.com/rust-lang/rust/issues/111540).

Don't know if it will fix our root issue
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=14875) but I don't see how to workaround all of that without this upstream work.

Quoted 6 lines

For cargo-c specific issues, I'll try to make things better (the TMPDIR in debug symbols) but I'm wondering
something, reproducibility is desirable for all generated artifacts or is it only for target ones ? I mean,
the cargo-c is used in native so I'm wondering if making it native only would be acceptable ?

Quoted 3 lines

That would certainly avoid this issue. It would also make on target testing of the functionality not possible.

The test I added is on the final binary generated (rust-c-lib-examble-bin), not cargo-c itself. So the target test would still be valid.

I tested a release build with a custom patch that strips debuginfo (strip="debug info" in Cargo.toml) and I do not have build path leaking in debug symbol anymore. I'll dig this up more and see if I can upstream that.

But the reproducible issue remain despite that.

Quoted 6 lines

Cheers,

Richard

#20 lore raw
Quoted 54 lines · Frédéric Martinsons

Le lun. 14 août 2023, 10:46, Richard Purdie
<[email protected]> a écrit :

Quoted 50 lines · Frédéric Martinsons
Quoted 31 lines
Quoted 19 lines · Richard Purdie
Quoted 9 lines · Frédéric Martinsons
Quoted 7 lines · Richard Purdie

and a reproducibility failure:

https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3355/steps/13/logs/stdio

which leads to:

Quoted 8 lines
Quoted 2 lines

http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230813-z_b2j3ha/packages/diff-html/

Argh, this makes me remember
of https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
Do you know if any of cargo based recipe is reproducible ? Should I add EXCLUDE_FROM_WORLD in cargo-c ?

Quoted 6 lines

At some point we're going to have to dive in and fix the reproducibility issues so I'm reluctant to take more patches with
that
set...

Quoted 11 lines

It will hardly be doable to correct the reproducibility of rust packages since rust/cargo
itself have issues upstream. I take a look and there are lot of open
issues
cargo:
https://github.com/rust-lang/cargo/issues?q=is%3Aissue+is%3Aopen+labe
l%3AA-reproducibility+
rust:
https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+label
%3AA-reproducibility

I believe there is one problematic reference left which causes problems
for core rust. Fixing that issue would be the best place to start and
then we can work from there.

Quoted 6 lines
Quoted 2 lines

We do already have a lot of good reproducibility pieces in the toolchain so not every upstream bug will apply to us.

I'm afraid I do not see what is your point, can you elaborate please? What is the "problematic reference left which causes problems for core rust" you refer to?

https://bugzilla.yoctoproject.org/show_bug.cgi?id=14875

There appeared to be one reference left in the binaries we couldn't track down. If we fix that one reference, at least the main rust recipe issues *should* be fixed.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

There was an RFC accepted back in may
(https://github.com/rust-lang/rfcs/pull/3127) and implementation progress have a tracking issue in cargo
(https://github.com/rust-lang/cargo/issues/12137) and in rust
(https://github.com/rust-lang/rust/issues/111540).

Don't know if it will fix our root issue
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=14875) but I don't
see how to workaround all of that without this upstream work.

For cargo-c specific issues, I'll try to make things better (the TMPDIR in debug symbols) but I'm wondering
something, reproducibility is  desirable for all generated artifacts
or is it only for target ones ?  I mean,
the cargo-c is used in native so I'm wondering if making it native
only would be acceptable ?

Quoted 6 lines
Quoted 2 lines

That would certainly avoid this issue. It would also make on target testing of the functionality not possible.

The test I added is on the final binary generated (rust-c-lib- examble-bin), not cargo-c itself. So the target test would still be valid.

Ok, good.

Quoted 4 lines

I tested a release build with a custom patch that strips debuginfo (strip="debug info" in Cargo.toml) and I do not have build path leaking in debug symbol anymore. I'll dig this up more and see if I can upstream that.

Debug symbols should be relocated correctly if we pass the correct compiler options. We usually prefer debug symbols to be present so we can package them into the -dbg packages.

Cheers,

Richard