Skip to contentSkip to conversation

Add 'maturin' python backend support

PATCH Superseded Tim Orling 4 participants 17 messages Dec 7, 2023 – Dec 14, 2023 2 messages not in the archive

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Tim Orling

author
#0 lore raw

cover Add 'maturin' python backend support

We are increasingly seeing Rust extensions for Python added in the ecosystem. The current best practices approach has moved from using 'setuptools-rust' to using 'maturin'.

* Add python3-maturin, including 'native' and 'nativesdk' support.
* Add a 'python_maturin' bbclass for the PEP-517 backend.
* Add recipetool support for the 'maturin' PEP-517 backend.
* Add a very basic 'maturin list-python' smoke test added to oeqa for both 'runtime' (-c testimage) and 'sdk' (-c testsdk).

The following changes since commit 70ad9b9b309575134834757468cb32c9e3b87448:

bitbake: hashserv: sqlite: Ensure sync propagates to database connections (2023-12-04 22:36:09 +0000)

are available in the Git repository at:

https://git.yoctoproject.org/poky-contrib timo/maturin_1.3.2
https://git.yoctoproject.org/poky-contrib/log/?h=timo/maturin_1.3.2

Tim Orling (6):
python3-setuptools-rust: BBCLASSEXTEND + nativesdk
python3-maturin: add v1.3.2
classes-recipe: add python_maturin.bbclass
recipetool: add python_maturin support
oe-selfest: add maturn runtime (testimage) test
python3-maturin: add sdk (testsdk) test case

 meta/classes-recipe/python_maturin.bbclass    |  17 +
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/lib/oeqa/runtime/cases/maturin.py        |  23 +
 meta/lib/oeqa/sdk/cases/maturin.py            |  33 +
 meta/lib/oeqa/selftest/cases/recipetool.py    |  34 ++
 .../python/python3-maturin-crates.inc         | 572 ++++++++++++++++++
 .../python/python3-maturin_1.3.2.bb           |  40 ++
 .../python/python3-setuptools-rust_1.8.1.bb   |   2 +-
 .../lib/recipetool/create_buildsys_python.py  |   1 +
 9 files changed, 722 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes-recipe/python_maturin.bbclass
 create mode 100644 meta/lib/oeqa/runtime/cases/maturin.py
 create mode 100644 meta/lib/oeqa/sdk/cases/maturin.py
 create mode 100644 meta/recipes-devtools/python/python3-maturin-crates.inc
 create mode 100644 meta/recipes-devtools/python/python3-maturin_1.3.2.bb
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH 1/6 python3-setuptools-rust: BBCLASSEXTEND + nativesdk

We would like to build python3-maturin for nativesdk use and nativesdk-python3-setuptools-rust is a dependency.

---
 meta/recipes-devtools/python/python3-setuptools-rust_1.8.1.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-setuptools-rust_1.8.1.bb +1 -1
index db5c6c449c3..a91ee23ca0c 100644
@@ -33,4 +33,4 @@ RDEPENDS:${PN} += " \
     python3-wheel \
 "
 
-BBCLASSEXTEND = "native"
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH 2/6 python3-maturin: add v1.3.2

In reply to Tim Orling

Build and publish crates with pyo3, rust-cpython, cffi and uniffi bindings as well as rust binaries as python packages.

This project is meant as a zero configuration replacement for setuptools-rust and milksnake. It supports building wheels for python 3.7+ on windows, linux, mac and freebsd, can upload them to pypi and has basic pypy and graalpy support.

https://github.com/pyo3/maturin

[YOCTO #15312]

---
 meta/conf/distro/include/maintainers.inc      |   1 +
 .../python/python3-maturin-crates.inc         | 572 ++++++++++++++++++
 .../python/python3-maturin_1.3.2.bb           |  40 ++
 3 files changed, 613 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-maturin-crates.inc
 create mode 100644 meta/recipes-devtools/python/python3-maturin_1.3.2.bb
meta/conf/distro/include/maintainers.inc +1 -0
index 2c59db56ac6..8fe1efd3288 100644
@@ -653,6 +653,7 @@ RECIPE_MAINTAINER:pn-python3-magic = "Joshua Watt <[email protected]>"
 RECIPE_MAINTAINER:pn-python3-mako = "Trevor Gamblin <[email protected]>"
 RECIPE_MAINTAINER:pn-python3-markdown = "Alexander Kanavin <[email protected]>"
 RECIPE_MAINTAINER:pn-python3-markupsafe = "Richard Purdie <[email protected]>"
+RECIPE_MAINTAINER:pn-python3-maturin = "Tim Orling <[email protected]>"
 RECIPE_MAINTAINER:pn-python3-more-itertools = "Tim Orling <[email protected]>"
 RECIPE_MAINTAINER:pn-python3-ndg-httpsclient = "Tim Orling <[email protected]>"
 RECIPE_MAINTAINER:pn-python3-numpy = "Trevor Gamblin <[email protected]>"
meta/recipes-devtools/python/python3-maturin-crates.inc new file +572 -0

572 lines not on this page · raw message

meta/recipes-devtools/python/python3-maturin_1.3.2.bb new file +40 -0
new file mode 100644
index 00000000000..4ab91024cf5
@@ -0,0 +1,40 @@
+SUMMARY = "Build and publish crates with pyo3, rust-cpython, cffi bindings and rust binaries as python packages"
+HOMEPAGE = "https://github.com/pyo3/maturin"
+SECTION = "devel/python"
+LICENSE = "MIT | Apache-2.0"
+LIC_FILES_CHKSUM = "file://license-apache;md5=1836efb2eb779966696f473ee8540542 \
+                    file://license-mit;md5=85fd3b67069cff784d98ebfc7d5c0797"
+
+SRC_URI[sha256sum] = "952d458b3dea01e9670582ac2dff3c522a34e491f08ed6376b270655a68c24af"
+
+S = "${WORKDIR}/maturin-${PV}"
+
+DEPENDS += "\
+    python3-setuptools-rust-native \
+    python3-semantic-version-native \
+    python3-setuptools-rust \
+"
+
+require ${BPN}-crates.inc
+
+inherit pypi cargo-update-recipe-crates python_pyo3 python_setuptools_build_meta
+
+do_configure() {
+    python_pyo3_do_configure
+    cargo_common_do_configure
+    python_pep517_do_configure
+}
+
+RDEPENDS:${PN} += "\
+    cargo \
+    python3-json \
+    rust \
+"
+
+RRECOMMENDS:${PN} += "\
+    python3-ensurepip \
+    python3-pip \
+    python3-venv \
+"
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH 3/6 classes-recipe: add python_maturin.bbclass

In reply to Tim Orling

maturin is a replacement for setuptools_rust and another "backend" for building python wheels.

[YOCTO #15312]

---
 meta/classes-recipe/python_maturin.bbclass | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 meta/classes-recipe/python_maturin.bbclass
meta/classes-recipe/python_maturin.bbclass new file +17 -0
new file mode 100644
index 00000000000..c39d6c6e37d
@@ -0,0 +1,17 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit python_pyo3 python_setuptools_build_meta
+
+DEPENDS += "python3-maturin-native"
+
+python_maturin_do_configure() {
+    python_pyo3_do_configure
+    cargo_common_do_configure
+    python_pep517_do_configure
+}
+
+EXPORT_FUNCTIONS do_configure
Signature · 1 line
2.34.1
author
#4 lore raw

PATCH 4/6 recipetool: add python_maturin support

In reply to Tim Orling

Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

[YOCTO #15312]

---
 meta/lib/oeqa/selftest/cases/recipetool.py    | 34 +++++++++++++++++++
 .../lib/recipetool/create_buildsys_python.py  |  1 +
 2 files changed, 35 insertions(+)
meta/lib/oeqa/selftest/cases/recipetool.py +34 -0
index 55cbba9ca74..ddcacaa73c7 100644
@@ -611,6 +611,40 @@ class RecipetoolCreateTests(RecipetoolBase):
 
         self._test_recipe_contents(recipefile, checkvars, inherits)
 
+    def test_recipetool_create_python3_pep517_maturin(self):
+        # This test require python 3.11 or above for the tomllib module
+        # or tomli module to be installed
+        try:
+            import tomllib
+        except ImportError:
+            try:
+                import tomli
+            except ImportError:
+                self.skipTest('Test requires python 3.11 or above for tomllib module or tomli module')
+
+        # Test creating python3 package from tarball (using maturin class)
+        temprecipe = os.path.join(self.tempdir, 'recipe')
+        os.makedirs(temprecipe)
+        pn = 'pydantic-core'
+        pv = '2.14.5'
+        recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+        srcuri = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-%s.tar.gz' % pv
+        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        self.assertTrue(os.path.isfile(recipefile))
+        checkvars = {}
+        checkvars['HOMEPAGE'] = 'https://github.com/pydantic/pydantic-core'
+        checkvars['LICENSE'] = set(['MIT'])
+        checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=ab599c188b4a314d2856b3a55030c75c'
+        checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-${PV}.tar.gz'
+        checkvars['SRC_URI[md5sum]'] = '1eb13c211147496c1c9484ff7f8ac438'
+        checkvars['SRC_URI[sha1sum]'] = '0803a731aa793f3eaf4d52f656d7300408ca0b36'
+        checkvars['SRC_URI[sha256sum]'] = '6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71'
+        checkvars['SRC_URI[sha384sum]'] = 'cca6d9f0ba72ff92e16aaf3d6c628b8df33bdf244cc39c9d59cff9d541f35d4e87ec136e60e4b040ad73c5c69b27cec6'
+        checkvars['SRC_URI[sha512sum]'] = 'e6c3081dfa58d9b59f8c33dfe80712d732ac3048a2f8240f9a0f5448d9bfc83e222c1e08b8bb49779b84f685c3bec3060cc935c113a818b1ab8aa955f3df2238'
+        inherits = ['python_maturin']
+
+        self._test_recipe_contents(recipefile, checkvars, inherits)
+
     def test_recipetool_create_github_tarball(self):
         # Basic test to ensure github URL mangling doesn't apply to release tarballs
         temprecipe = os.path.join(self.tempdir, 'recipe')
scripts/lib/recipetool/create_buildsys_python.py +1 -0
index 9312e4abf13..9bebb3061b1 100644
@@ -668,6 +668,7 @@ class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):
         "poetry.core.masonry.api": "python_poetry_core",
         "flit_core.buildapi": "python_flit_core",
         "hatchling.build": "python_hatchling",
+        "maturin": "python_maturin",
     }
 
     # setuptools.build_meta and flit declare project metadata into the "project" section of pyproject.toml
Signature · 1 line
2.34.1
#5 lore raw

RE: [OE-core] [PATCH 4/6] recipetool: add python_maturin support

Quoted 46 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Tim Orling
Sent: den 8 december 2023 00:13
To: [email protected]
Subject: [OE-core] [PATCH 4/6] recipetool: add python_maturin support

Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

[YOCTO #15312]

---
 meta/lib/oeqa/selftest/cases/recipetool.py    | 34 +++++++++++++++++++
 .../lib/recipetool/create_buildsys_python.py  |  1 +
 2 files changed, 35 insertions(+)
meta/lib/oeqa/selftest/cases/recipetool.py +20 -0
index 55cbba9ca74..ddcacaa73c7 100644
@@ -611,6 +611,40 @@ class RecipetoolCreateTests(RecipetoolBase):
 
         self._test_recipe_contents(recipefile, checkvars, inherits)
 
+    def test_recipetool_create_python3_pep517_maturin(self):
+        # This test require python 3.11 or above for the tomllib module
+        # or tomli module to be installed
+        try:
+            import tomllib
+        except ImportError:
+            try:
+                import tomli
+            except ImportError:
+                self.skipTest('Test requires python 3.11 or above for tomllib module or tomli module')
+
+        # Test creating python3 package from tarball (using maturin class)
+        temprecipe = os.path.join(self.tempdir, 'recipe')
+        os.makedirs(temprecipe)
+        pn = 'pydantic-core'
+        pv = '2.14.5'
+        recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+        srcuri = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-%s.tar.gz' % pv
+        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        self.assertTrue(os.path.isfile(recipefile))

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 643

Quoted 10 lines
meta/lib/oeqa/selftest/cases/recipetool.py +10 -0
+        checkvars = {}
+        checkvars['HOMEPAGE'] = 'https://github.com/pydantic/pydantic-core'
+        checkvars['LICENSE'] = set(['MIT'])
+        checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=ab599c188b4a314d2856b3a55030c75c'
+        checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-${PV}.tar.gz'
+        checkvars['SRC_URI[md5sum]'] = '1eb13c211147496c1c9484ff7f8ac438'
+        checkvars['SRC_URI[sha1sum]'] = '0803a731aa793f3eaf4d52f656d7300408ca0b36'
+        checkvars['SRC_URI[sha256sum]'] = '6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71'
+        checkvars['SRC_URI[sha384sum]'] = 'cca6d9f0ba72ff92e16aaf3d6c628b8df33bdf244cc39c9d59cff9d541f35d4e87ec136e6 0e4b040ad73c5c69b27cec6'
+        checkvars['SRC_URI[sha512sum]'] = 'e6c3081dfa58d9b59f8c33dfe80712d732ac3048a2f8240f9a0f5448d9bfc83e222c1e08b8bb49779b84f685c3bec3060cc935c113a818b1ab8aa955f3df2238'

After my recent changes to recipetool, only the sha256sum shall be present here.

Quoted 19 lines
meta/lib/oeqa/selftest/cases/recipetool.py +4 -0
+        inherits = ['python_maturin']
+
+        self._test_recipe_contents(recipefile, checkvars, inherits)
+
     def test_recipetool_create_github_tarball(self):
         # Basic test to ensure github URL mangling doesn't apply to release tarballs
         temprecipe = os.path.join(self.tempdir, 'recipe')
scripts/lib/recipetool/create_buildsys_python.py +0 -0
index 9312e4abf13..9bebb3061b1 100644
@@ -668,6 +668,7 @@ class

PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):
"poetry.core.masonry.api": "python_poetry_core", "flit_core.buildapi": "python_flit_core", "hatchling.build": "python_hatchling",
+ "maturin": "python_maturin",
}

Quoted 3 lines

# setuptools.build_meta and flit declare project metadata into the "project" section of pyproject.toml
--
2.34.1

//Peter

author
#6 lore raw

PATCH v2 4/6 recipetool: add python_maturin support

In reply to Tim Orling

Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

---
Changes in v2:
- drop all SRC_URI checksums except sha256sum (Peter's change)
* requires "[PATCH v2] recipetool: pypi: do not clobber SRC_URI checksums"
- drop SRC_URI (Julien's change)
- add "inherit pypi" (Julien's change)

 meta/lib/oeqa/selftest/cases/recipetool.py    | 29 +++++++++++++++++++
 .../lib/recipetool/create_buildsys_python.py  |  1 +
 2 files changed, 30 insertions(+)
meta/lib/oeqa/selftest/cases/recipetool.py +29 -0
index d10d61bc5b0..8d39f639acf 100644
@@ -663,6 +663,35 @@ class RecipetoolCreateTests(RecipetoolBase):
 
         self._test_recipe_contents(recipefile, checkvars, inherits)
 
+    def test_recipetool_create_python3_pep517_maturin(self):
+        # This test require python 3.11 or above for the tomllib module
+        # or tomli module to be installed
+        try:
+            import tomllib
+        except ImportError:
+            try:
+                import tomli
+            except ImportError:
+                self.skipTest('Test requires python 3.11 or above for tomllib module or tomli module')
+
+        # Test creating python3 package from tarball (using maturin class)
+        temprecipe = os.path.join(self.tempdir, 'recipe')
+        os.makedirs(temprecipe)
+        pn = 'pydantic-core'
+        pv = '2.14.5'
+        recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+        srcuri = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-%s.tar.gz' % pv
+        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        self.assertTrue(os.path.isfile(recipefile))
+        checkvars = {}
+        checkvars['HOMEPAGE'] = 'https://github.com/pydantic/pydantic-core'
+        checkvars['LICENSE'] = set(['MIT'])
+        checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=ab599c188b4a314d2856b3a55030c75c'
+        checkvars['SRC_URI[sha256sum]'] = '6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71'
+        inherits = ['python_maturin', 'pypi']
+
+        self._test_recipe_contents(recipefile, checkvars, inherits)
+
     def test_recipetool_create_github_tarball(self):
         # Basic test to ensure github URL mangling doesn't apply to release tarballs
         temprecipe = os.path.join(self.tempdir, 'recipe')
scripts/lib/recipetool/create_buildsys_python.py +1 -0
index a2af41fdda8..60c59034504 100644
@@ -735,6 +735,7 @@ class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):
         "poetry.core.masonry.api": "python_poetry_core",
         "flit_core.buildapi": "python_flit_core",
         "hatchling.build": "python_hatchling",
+        "maturin": "python_maturin",
     }
 
     # setuptools.build_meta and flit declare project metadata into the "project" section of pyproject.toml
Signature · 1 line
2.34.1

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

Re: [OE-core] [PATCH v2 4/6] recipetool: add python_maturin support

Quoted 5 lines · Alexandre Belloni via lists.openembedded.org

Hello,

maturin is not reproducible:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231208-rfbproj4/packages/diff-html/

Probably related:

WARNING: python3-maturin-1.3.2-r0 do_package_qa: QA Issue: File /usr/bin/.debug/maturin in package python3-maturin-dbg contains reference to TMPDIR
https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/8267/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6018/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/52/builds/8141/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/108/builds/5453/steps/11/logs/stdio

Quoted 94 lines · Tim Orling
Quoted 75 lines

Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

---
Changes in v2:
- drop all SRC_URI checksums except sha256sum (Peter's change)
* requires "[PATCH v2] recipetool: pypi: do not clobber SRC_URI checksums"
- drop SRC_URI (Julien's change)
- add "inherit pypi" (Julien's change)

 meta/lib/oeqa/selftest/cases/recipetool.py    | 29 +++++++++++++++++++
 .../lib/recipetool/create_buildsys_python.py  |  1 +
 2 files changed, 30 insertions(+)
meta/lib/oeqa/selftest/cases/recipetool.py +29 -0
index d10d61bc5b0..8d39f639acf 100644
@@ -663,6 +663,35 @@ class RecipetoolCreateTests(RecipetoolBase):
 
         self._test_recipe_contents(recipefile, checkvars, inherits)
 
+    def test_recipetool_create_python3_pep517_maturin(self):
+        # This test require python 3.11 or above for the tomllib module
+        # or tomli module to be installed
+        try:
+            import tomllib
+        except ImportError:
+            try:
+                import tomli
+            except ImportError:
+                self.skipTest('Test requires python 3.11 or above for tomllib module or tomli module')
+
+        # Test creating python3 package from tarball (using maturin class)
+        temprecipe = os.path.join(self.tempdir, 'recipe')
+        os.makedirs(temprecipe)
+        pn = 'pydantic-core'
+        pv = '2.14.5'
+        recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+        srcuri = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-%s.tar.gz' % pv
+        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        self.assertTrue(os.path.isfile(recipefile))
+        checkvars = {}
+        checkvars['HOMEPAGE'] = 'https://github.com/pydantic/pydantic-core'
+        checkvars['LICENSE'] = set(['MIT'])
+        checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=ab599c188b4a314d2856b3a55030c75c'
+        checkvars['SRC_URI[sha256sum]'] = '6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71'
+        inherits = ['python_maturin', 'pypi']
+
+        self._test_recipe_contents(recipefile, checkvars, inherits)
+
     def test_recipetool_create_github_tarball(self):
         # Basic test to ensure github URL mangling doesn't apply to release tarballs
         temprecipe = os.path.join(self.tempdir, 'recipe')
scripts/lib/recipetool/create_buildsys_python.py +1 -0
index a2af41fdda8..60c59034504 100644
@@ -735,6 +735,7 @@ class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):
         "poetry.core.masonry.api": "python_poetry_core",
         "flit_core.buildapi": "python_flit_core",
         "hatchling.build": "python_hatchling",
+        "maturin": "python_maturin",
     }
 
     # setuptools.build_meta and flit declare project metadata into the "project" section of pyproject.toml
Signature · 1 line
2.34.1
Signature · 12 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192097): https://lists.openembedded.org/g/openembedded-core/message/192097
Mute This Topic: https://lists.openembedded.org/mt/103071823/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
author
#9 lore raw

Re: [OE-core] [PATCH v2 4/6] recipetool: add python_maturin support

Quoted 18 lines · Alexandre Belloni
Quoted 5 lines · Alexandre Belloni via lists.openembedded.org

Hello,

maturin is not reproducible:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231208-rfbproj4/packages/diff-html/

Probably related:

WARNING: python3-maturin-1.3.2-r0 do_package_qa: QA Issue: File /usr/bin/.debug/maturin in package python3-maturin-dbg contains reference to TMPDIR

https://autobuilder.yoctoproject.org/typhoon/#/builders/40/builds/8267/steps/14/logs/stdio

Quoted 7 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/81/builds/6018/steps/12/logs/stdio

https://autobuilder.yoctoproject.org/typhoon/#/builders/52/builds/8141/steps/11/logs/stdio

https://autobuilder.yoctoproject.org/typhoon/#/builders/108/builds/5453/steps/11/logs/stdio

I think at least this part is sorted:

https://github.com/alexcrichton/bzip2-rs/pull/101

Thank you to khem for making me realize the options passed to the ‘cc’ crate were the issue in this case. (The Makefile is not used, nor are CFLAGS from the environment)

The reordering bits might be a different problem. If anybody has any thoughts on that I’m all ears.

Quoted 120 lines
Quoted 12 lines · Tim Orling
Quoted 9 lines

Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

---
Changes in v2:
- drop all SRC_URI checksums except sha256sum (Peter's change)
* requires "[PATCH v2] recipetool: pypi: do not clobber SRC_URI

checksums"

Quoted 8 lines

Review

Quoted 8 lines

- drop SRC_URI (Julien's change)
- add "inherit pypi" (Julien's change)

 meta/lib/oeqa/selftest/cases/recipetool.py    | 29 +++++++++++++++++++
 .../lib/recipetool/create_buildsys_python.py  |  1 +
 2 files changed, 30 insertions(+)
(quoted diff) +0 -0

b/meta/lib/oeqa/selftest/cases/recipetool.py

Quoted 9 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 667

Quoted 9 lines

index d10d61bc5b0..8d39f639acf 100644

meta/lib/oeqa/selftest/cases/recipetool.py +2 -0
@@ -663,6 +663,35 @@ class RecipetoolCreateTests(RecipetoolBase):
 
         self._test_recipe_contents(recipefile, checkvars, inherits)
 
+    def test_recipetool_create_python3_pep517_maturin(self):
+        # This test require python 3.11 or above for the tomllib

module

Quoted 8 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase):

Quoted 8 lines
meta/lib/oeqa/selftest/cases/recipetool.py +8 -0
+        # or tomli module to be installed
+        try:
+            import tomllib
+        except ImportError:
+            try:
+                import tomli
+            except ImportError:
+                self.skipTest('Test requires python 3.11 or above for

tomllib module or tomli module')

Quoted 2 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 677

Quoted 2 lines
meta/lib/oeqa/selftest/cases/recipetool.py +2 -0
+
+        # Test creating python3 package from tarball (using maturin

class)

Quoted 5 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase):

Quoted 5 lines
meta/lib/oeqa/selftest/cases/recipetool.py +5 -0
+        temprecipe = os.path.join(self.tempdir, 'recipe')
+        os.makedirs(temprecipe)
+        pn = 'pydantic-core'
+        pv = '2.14.5'
+        recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' %

(pn, pv))

Quoted 1 line

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 683

Quoted 1 line
meta/lib/oeqa/selftest/cases/recipetool.py +1 -0
+        srcuri = '

https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-%s.tar.gz'
% pv

Quoted 1 line

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 684

Quoted 1 line
meta/lib/oeqa/selftest/cases/recipetool.py +1 -0
+        result = runCmd('recipetool create -o %s %s' % (temprecipe,

srcuri))

Quoted 3 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase):

Quoted 3 lines
meta/lib/oeqa/selftest/cases/recipetool.py +3 -0
+        self.assertTrue(os.path.isfile(recipefile))
+        checkvars = {}
+        checkvars['HOMEPAGE'] = '

https://github.com/pydantic/pydantic-core'

Quoted 2 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 689

Quoted 2 lines
meta/lib/oeqa/selftest/cases/recipetool.py +2 -0
+        checkvars['LICENSE'] = set(['MIT'])
+        checkvars['LIC_FILES_CHKSUM'] =

'file://LICENSE;md5=ab599c188b4a314d2856b3a55030c75c'

Quoted 1 line

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase): · line 690

Quoted 1 line
meta/lib/oeqa/selftest/cases/recipetool.py +1 -0
+        checkvars['SRC_URI[sha256sum]'] =

'6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71'

Quoted 6 lines

meta/lib/oeqa/selftest/cases/recipetool.py class RecipetoolCreateTests(RecipetoolBase):

Quoted 6 lines
meta/lib/oeqa/selftest/cases/recipetool.py +4 -0
+        inherits = ['python_maturin', 'pypi']
+
+        self._test_recipe_contents(recipefile, checkvars, inherits)
+
     def test_recipetool_create_github_tarball(self):
         # Basic test to ensure github URL mangling doesn't apply to

release tarballs

Quoted 2 lines

Review

Quoted 2 lines
meta/lib/oeqa/selftest/cases/recipetool.py +0 -0
         temprecipe = os.path.join(self.tempdir, 'recipe')
(quoted diff) +0 -0

b/scripts/lib/recipetool/create_buildsys_python.py

Quoted 4 lines

scripts/lib/recipetool/create_buildsys_python.py class

Quoted 4 lines

index a2af41fdda8..60c59034504 100644

scripts/lib/recipetool/create_buildsys_python.py +0 -0
@@ -735,6 +735,7 @@ class

PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):

Quoted 7 lines

scripts/lib/recipetool/create_buildsys_python.py class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):

Quoted 7 lines
scripts/lib/recipetool/create_buildsys_python.py +1 -0
         "poetry.core.masonry.api": "python_poetry_core",
         "flit_core.buildapi": "python_flit_core",
         "hatchling.build": "python_hatchling",
+        "maturin": "python_maturin",
     }
 
     # setuptools.build_meta and flit declare project metadata into

the "project" section of pyproject.toml

Quoted 16 lines
Quoted 7 lines

--
2.34.1

--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Quoted 3 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#192097):

https://lists.openembedded.org/g/openembedded-core/message/192097

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/103071823/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

author
#10 lore raw

PATCH 5/6 oe-selfest: add maturn runtime (testimage) test

In reply to Tim Orling

Basic smoke test for maturin to test the 'maturin list-python' case.

[YOCTO #15312]

---
 meta/lib/oeqa/runtime/cases/maturin.py | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/cases/maturin.py
meta/lib/oeqa/runtime/cases/maturin.py new file +23 -0
new file mode 100644
index 00000000000..b9a3b4acbcd
@@ -0,0 +1,23 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import os
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class MaturinTest(OERuntimeTestCase):
+    @OETestDepends(['ssh.SSHTest.test_ssh', 'python.PythonTest.test_python3'])
+    @OEHasPackage(['python3-maturin'])
+    def test_maturin_list_python(self):
+        status, output = self.target.run("maturin list-python")
+        self.assertEqual(status, 0)
+        _, py_major = self.target.run("python3 -c 'import sys; print(sys.version_info.major)'")
+        _, py_minor = self.target.run("python3 -c 'import sys; print(sys.version_info.minor)'")
+        python_version = "%s.%s" % (py_major, py_minor)
+        self.assertEqual(output, "🐍 1 python interpreter found:\n"
+                                 " - CPython %s at /usr/bin/python%s" % (python_version, python_version))
Signature · 1 line
2.34.1
#11 lore raw

Patchtest results for [PATCH 5/6] oe-selfest: add maturn runtime (testimage) test

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/5-6-oe-selfest-add-maturn-runtime-testimage-test.patch

FAIL: test mbox format: Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am (test_mbox.TestMbox.test_mbox_format)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: Python-unidiff parse error (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE tag format: Parse error Hunk is shorter than expected (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: Parse error Hunk is shorter than expected (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: Parse error Hunk is shorter than expected (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: Python-unidiff parse error (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

author
#12 lore raw

PATCH 6/6 python3-maturin: add sdk (testsdk) test case

In reply to Tim Orling

We expect 'maturin' will be used in SDKs, so it makes sense to also test it in the testsdk environment.

To run this test case, you can add the following to local.conf:
TOOLCHAIN_HOST_TASK:append = " nativesdk-python3-maturin"

And the build and test the SDK:
bitbake -c populate_sdk core-image-full-cmdline
bitbake -c testsdk core-image-full-cmdline

You can substitute a different image recipe for "core-image-full-cmdline"

[YOCTO #15312]

---
 meta/lib/oeqa/sdk/cases/maturin.py | 33 ++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
 create mode 100644 meta/lib/oeqa/sdk/cases/maturin.py
meta/lib/oeqa/sdk/cases/maturin.py new file +33 -0
new file mode 100644
index 00000000000..14245bc36e5
@@ -0,0 +1,33 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import unittest
+from oeqa.sdk.case import OESDKTestCase
+
+from oeqa.utils.subprocesstweak import errors_have_output
+
+errors_have_output()
+
+
+class MaturinTest(OESDKTestCase):
+    def setUp(self):
+        if not (
+            self.tc.hasHostPackage("nativesdk-python3-maturin")
+            or self.tc.hasHostPackage("python3-maturin-native")
+        ):
+            raise unittest.SkipTest("No python3-maturin package in the SDK")
+
+    def test_maturin_list_python(self):
+        py_major = self._run("python3 -c 'import sys; print(sys.version_info.major)'")
+        py_minor = self._run("python3 -c 'import sys; print(sys.version_info.minor)'")
+        python_version = "%s.%s" % (py_major.strip(), py_minor.strip())
+        cmd = "maturin list-python"
+        output = self._run(cmd)
+        self.assertRegex(output, r"^🐍 1 python interpreter found:\n")
+        self.assertRegex(
+            output,
+            r" - CPython %s (.+)/usr/bin/python%s$" % (python_version, python_version),
+        )
Signature · 1 line
2.34.1
#13 lore raw

Patchtest results for [PATCH 6/6] python3-maturin: add sdk (testsdk) test case

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/6-6-python3-maturin-add-sdk-testsdk-test-case.patch

FAIL: test mbox format: Series has malformed diff lines. Create the series again using git-format-patch and ensure it applies using git am (test_mbox.TestMbox.test_mbox_format)

PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: Python-unidiff parse error (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE tag format: Parse error Hunk is shorter than expected (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: Parse error Hunk is shorter than expected (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: Parse error Hunk is shorter than expected (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: Python-unidiff parse error (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

#14 lore raw

Re: [OE-core] [PATCH v2 4/6] recipetool: add python_maturin support

In reply to Tim Orling

Hello,

maturin is not reproducible:

https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231208-rfbproj4/packages/diff-html/

Quoted 81 lines · Tim Orling

Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

---
Changes in v2:
- drop all SRC_URI checksums except sha256sum (Peter's change)
* requires "[PATCH v2] recipetool: pypi: do not clobber SRC_URI checksums"
- drop SRC_URI (Julien's change)
- add "inherit pypi" (Julien's change)

 meta/lib/oeqa/selftest/cases/recipetool.py    | 29 +++++++++++++++++++
 .../lib/recipetool/create_buildsys_python.py  |  1 +
 2 files changed, 30 insertions(+)
meta/lib/oeqa/selftest/cases/recipetool.py +29 -0
index d10d61bc5b0..8d39f639acf 100644
@@ -663,6 +663,35 @@ class RecipetoolCreateTests(RecipetoolBase):
 
         self._test_recipe_contents(recipefile, checkvars, inherits)
 
+    def test_recipetool_create_python3_pep517_maturin(self):
+        # This test require python 3.11 or above for the tomllib module
+        # or tomli module to be installed
+        try:
+            import tomllib
+        except ImportError:
+            try:
+                import tomli
+            except ImportError:
+                self.skipTest('Test requires python 3.11 or above for tomllib module or tomli module')
+
+        # Test creating python3 package from tarball (using maturin class)
+        temprecipe = os.path.join(self.tempdir, 'recipe')
+        os.makedirs(temprecipe)
+        pn = 'pydantic-core'
+        pv = '2.14.5'
+        recipefile = os.path.join(temprecipe, 'python3-%s_%s.bb' % (pn, pv))
+        srcuri = 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-%s.tar.gz' % pv
+        result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
+        self.assertTrue(os.path.isfile(recipefile))
+        checkvars = {}
+        checkvars['HOMEPAGE'] = 'https://github.com/pydantic/pydantic-core'
+        checkvars['LICENSE'] = set(['MIT'])
+        checkvars['LIC_FILES_CHKSUM'] = 'file://LICENSE;md5=ab599c188b4a314d2856b3a55030c75c'
+        checkvars['SRC_URI[sha256sum]'] = '6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71'
+        inherits = ['python_maturin', 'pypi']
+
+        self._test_recipe_contents(recipefile, checkvars, inherits)
+
     def test_recipetool_create_github_tarball(self):
         # Basic test to ensure github URL mangling doesn't apply to release tarballs
         temprecipe = os.path.join(self.tempdir, 'recipe')
scripts/lib/recipetool/create_buildsys_python.py +1 -0
index a2af41fdda8..60c59034504 100644
@@ -735,6 +735,7 @@ class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler):
         "poetry.core.masonry.api": "python_poetry_core",
         "flit_core.buildapi": "python_flit_core",
         "hatchling.build": "python_hatchling",
+        "maturin": "python_maturin",
     }
 
     # setuptools.build_meta and flit declare project metadata into the "project" section of pyproject.toml
Signature · 11 lines
2.34.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192096): https://lists.openembedded.org/g/openembedded-core/message/192096
Mute This Topic: https://lists.openembedded.org/mt/103067787/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 52 lines · Tim Orling

Basic smoke test for maturin to test the 'maturin list-python' case.

[YOCTO #15312]

---
 meta/lib/oeqa/runtime/cases/maturin.py | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/cases/maturin.py
meta/lib/oeqa/runtime/cases/maturin.py new file +23 -0
new file mode 100644
index 00000000000..b9a3b4acbcd
@@ -0,0 +1,23 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import os
+
+from oeqa.runtime.case import OERuntimeTestCase
+from oeqa.core.decorator.depends import OETestDepends
+from oeqa.runtime.decorator.package import OEHasPackage
+
+class MaturinTest(OERuntimeTestCase):
+    @OETestDepends(['ssh.SSHTest.test_ssh', 'python.PythonTest.test_python3'])
+    @OEHasPackage(['python3-maturin'])
+    def test_maturin_list_python(self):
+        status, output = self.target.run("maturin list-python")
+        self.assertEqual(status, 0)
+        _, py_major = self.target.run("python3 -c 'import sys; print(sys.version_info.major)'")
+        _, py_minor = self.target.run("python3 -c 'import sys; print(sys.version_info.minor)'")
+        python_version = "%s.%s" % (py_major, py_minor)
+        self.assertEqual(output, "🐍 1 python interpreter found:\n"
+                                 " - CPython %s at /usr/bin/python%s" % (python_version, python_version))
Signature · 11 lines
2.34.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191986): https://lists.openembedded.org/g/openembedded-core/message/191986
Mute This Topic: https://lists.openembedded.org/mt/103045102/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 72 lines · Tim Orling

We expect 'maturin' will be used in SDKs, so it makes sense to also test it in the testsdk environment.

To run this test case, you can add the following to local.conf:
TOOLCHAIN_HOST_TASK:append = " nativesdk-python3-maturin"

And the build and test the SDK:
bitbake -c populate_sdk core-image-full-cmdline
bitbake -c testsdk core-image-full-cmdline

You can substitute a different image recipe for "core-image-full-cmdline"

[YOCTO #15312]

---
 meta/lib/oeqa/sdk/cases/maturin.py | 33 ++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
 create mode 100644 meta/lib/oeqa/sdk/cases/maturin.py
meta/lib/oeqa/sdk/cases/maturin.py new file +33 -0
new file mode 100644
index 00000000000..14245bc36e5
@@ -0,0 +1,33 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+import unittest
+from oeqa.sdk.case import OESDKTestCase
+
+from oeqa.utils.subprocesstweak import errors_have_output
+
+errors_have_output()
+
+
+class MaturinTest(OESDKTestCase):
+    def setUp(self):
+        if not (
+            self.tc.hasHostPackage("nativesdk-python3-maturin")
+            or self.tc.hasHostPackage("python3-maturin-native")
+        ):
+            raise unittest.SkipTest("No python3-maturin package in the SDK")
+
+    def test_maturin_list_python(self):
+        py_major = self._run("python3 -c 'import sys; print(sys.version_info.major)'")
+        py_minor = self._run("python3 -c 'import sys; print(sys.version_info.minor)'")
+        python_version = "%s.%s" % (py_major.strip(), py_minor.strip())
+        cmd = "maturin list-python"
+        output = self._run(cmd)
+        self.assertRegex(output, r"^🐍 1 python interpreter found:\n")
+        self.assertRegex(
+            output,
+            r" - CPython %s (.+)/usr/bin/python%s$" % (python_version, python_version),
+        )
Signature · 11 lines
2.34.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191987): https://lists.openembedded.org/g/openembedded-core/message/191987
Mute This Topic: https://lists.openembedded.org/mt/103045103/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#15 lore raw
In reply to Tim Orling

Hello Tim,

I got this failure, can you check?

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6166/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6159/steps/14/logs/stdio

2023-12-08 22:03:18,822 - oe-selftest - INFO - FAIL: recipetool.RecipetoolCreateTests.test_recipetool_create_python3_pep517_maturin (subunit.RemotedTestCase)
2023-12-08 22:03:18,822 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-12-08 22:03:18,823 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/recipetool.py", line 698, in test_recipetool_create_python3_pep517_maturin
    self._test_recipe_contents(recipefile, checkvars, inherits)
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 178, in _test_recipe_contents
    self.assertEqual(missingvars, {}, 'Some expected variables not found in recipe: %s' % checkvars)
  File "/usr/lib64/python3.11/unittest/case.py", line 873, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib64/python3.11/unittest/case.py", line 1172, in assertDictEqual
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib64/python3.11/unittest/case.py", line 703, in fail
    raise self.failureException(msg)

AssertionError: {'SRC_URI': 'https://files.pythonhosted.or[586 chars]238'} != {}
Diff is 656 characters long. Set self.maxDiff to None to see it. : Some expected variables not found in recipe: {'SRC_URI': 'https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-${PV}.tar.gz', 'SRC_URI[md5sum]': '1eb13c211147496c1c9484ff7f8ac438', 'SRC_URI[sha1sum]': '0803a731aa793f3eaf4d52f656d7300408ca0b36', 'SRC_URI[sha256sum]': '6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71', 'SRC_URI[sha384sum]': 'cca6d9f0ba72ff92e16aaf3d6c628b8df33bdf244cc39c9d59cff9d541f35d4e87ec136e60e4b040ad73c5c69b27cec6', 'SRC_URI[sha512sum]': 'e6c3081dfa58d9b59f8c33dfe80712d732ac3048a2f8240f9a0f5448d9bfc83e222c1e08b8bb49779b84f685c3bec3060cc935c113a818b1ab8aa955f3df2238'}

Quoted 56 lines · Tim Orling

We are increasingly seeing Rust extensions for Python added in the ecosystem. The current best practices approach has moved from using 'setuptools-rust' to using 'maturin'.

* Add python3-maturin, including 'native' and 'nativesdk' support.
* Add a 'python_maturin' bbclass for the PEP-517 backend.
* Add recipetool support for the 'maturin' PEP-517 backend.
* Add a very basic 'maturin list-python' smoke test added to oeqa for both 'runtime' (-c testimage) and 'sdk' (-c testsdk).

The following changes since commit 70ad9b9b309575134834757468cb32c9e3b87448:

bitbake: hashserv: sqlite: Ensure sync propagates to database connections (2023-12-04 22:36:09 +0000)

are available in the Git repository at:

https://git.yoctoproject.org/poky-contrib timo/maturin_1.3.2
https://git.yoctoproject.org/poky-contrib/log/?h=timo/maturin_1.3.2

Tim Orling (6):
python3-setuptools-rust: BBCLASSEXTEND + nativesdk
python3-maturin: add v1.3.2
classes-recipe: add python_maturin.bbclass
recipetool: add python_maturin support
oe-selfest: add maturn runtime (testimage) test
python3-maturin: add sdk (testsdk) test case

 meta/classes-recipe/python_maturin.bbclass    |  17 +
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/lib/oeqa/runtime/cases/maturin.py        |  23 +
 meta/lib/oeqa/sdk/cases/maturin.py            |  33 +
 meta/lib/oeqa/selftest/cases/recipetool.py    |  34 ++
 .../python/python3-maturin-crates.inc         | 572 ++++++++++++++++++
 .../python/python3-maturin_1.3.2.bb           |  40 ++
 .../python/python3-setuptools-rust_1.8.1.bb   |   2 +-
 .../lib/recipetool/create_buildsys_python.py  |   1 +
 9 files changed, 722 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes-recipe/python_maturin.bbclass
 create mode 100644 meta/lib/oeqa/runtime/cases/maturin.py
 create mode 100644 meta/lib/oeqa/sdk/cases/maturin.py
 create mode 100644 meta/recipes-devtools/python/python3-maturin-crates.inc
 create mode 100644 meta/recipes-devtools/python/python3-maturin_1.3.2.bb
Signature · 11 lines
2.34.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191981): https://lists.openembedded.org/g/openembedded-core/message/191981
Mute This Topic: https://lists.openembedded.org/mt/103045095/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
author
#16 lore raw
Quoted 10 lines · Alexandre Belloni

Hello Tim,

I got this failure, can you check?

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6166/steps/14/logs/stdio

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6159/steps/14/logs/stdio

this is from Peter’s change which had not been merged when I sent this

series.

Quoted 111 lines
2023-12-08 22:03:18,822 - oe-selftest - INFO - FAIL:
recipetool.RecipetoolCreateTests.test_recipetool_create_python3_pep517_maturin
(subunit.RemotedTestCase)
2023-12-08 22:03:18,822 - oe-selftest - INFO -
----------------------------------------------------------------------
2023-12-08 22:03:18,823 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call
last):
File
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/recipetool.py",
line 698, in test_recipetool_create_python3_pep517_maturin
self._test_recipe_contents(recipefile, checkvars, inherits)
File
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py",
line 178, in _test_recipe_contents
self.assertEqual(missingvars, {}, 'Some expected variables not found
in recipe: %s' % checkvars)
File "/usr/lib64/python3.11/unittest/case.py", line 873, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib64/python3.11/unittest/case.py", line 1172, in
assertDictEqual
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib64/python3.11/unittest/case.py", line 703, in fail
raise self.failureException(msg)
AssertionError: {'SRC_URI': 'https://files.pythonhosted.or[586
chars]238'} != {}
Diff is 656 characters long. Set self.maxDiff to None to see it. : Some
expected variables not found in recipe: {'SRC_URI': '
https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-${PV}.tar.gz',
'SRC_URI[md5sum]': '1eb13c211147496c1c9484ff7f8ac438', 'SRC_URI[sha1sum]':
'0803a731aa793f3eaf4d52f656d7300408ca0b36', 'SRC_URI[sha256sum]':
'6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71',
'SRC_URI[sha384sum]':
'cca6d9f0ba72ff92e16aaf3d6c628b8df33bdf244cc39c9d59cff9d541f35d4e87ec136e60e4b040ad73c5c69b27cec6', 'SRC_URI[sha512sum]':
'e6c3081dfa58d9b59f8c33dfe80712d732ac3048a2f8240f9a0f5448d9bfc83e222c1e08b8bb49779b84f685c3bec3060cc935c113a818b1ab8aa955f3df2238'}

Quoted 11 lines · Tim Orling

We are increasingly seeing Rust extensions for Python added in the ecosystem. The current best practices approach has moved from using 'setuptools-rust' to using 'maturin'.

* Add python3-maturin, including 'native' and 'nativesdk' support.
* Add a 'python_maturin' bbclass for the PEP-517 backend.
* Add recipetool support for the 'maturin' PEP-517 backend.
* Add a very basic 'maturin list-python' smoke test added to oeqa for both 'runtime' (-c testimage) and 'sdk' (-c testsdk).

The following changes since commit

70ad9b9b309575134834757468cb32c9e3b87448:

Quoted 2 lines

bitbake: hashserv: sqlite: Ensure sync propagates to database

connections (2023-12-04 22:36:09 +0000)

Quoted 14 lines

are available in the Git repository at:

https://git.yoctoproject.org/poky-contrib timo/maturin_1.3.2
https://git.yoctoproject.org/poky-contrib/log/?h=timo/maturin_1.3.2

Tim Orling (6):
python3-setuptools-rust: BBCLASSEXTEND + nativesdk
python3-maturin: add v1.3.2
classes-recipe: add python_maturin.bbclass
recipetool: add python_maturin support
oe-selfest: add maturn runtime (testimage) test
python3-maturin: add sdk (testsdk) test case

Quoted 14 lines
 meta/classes-recipe/python_maturin.bbclass    |  17 +
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/lib/oeqa/runtime/cases/maturin.py        |  23 +
 meta/lib/oeqa/sdk/cases/maturin.py            |  33 +
 meta/lib/oeqa/selftest/cases/recipetool.py    |  34 ++
 .../python/python3-maturin-crates.inc         | 572 ++++++++++++++++++
 .../python/python3-maturin_1.3.2.bb           |  40 ++
 .../python/python3-setuptools-rust_1.8.1.bb   |   2 +-
 .../lib/recipetool/create_buildsys_python.py  |   1 +
 9 files changed, 722 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes-recipe/python_maturin.bbclass
 create mode 100644 meta/lib/oeqa/runtime/cases/maturin.py
 create mode 100644 meta/lib/oeqa/sdk/cases/maturin.py
 create mode 100644

meta/recipes-devtools/python/python3-maturin-crates.inc

Quoted 1 line
 create mode 100644 meta/recipes-devtools/python/

python3-maturin_1.3.2.bb

Quoted 9 lines

--
2.34.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#191981):

https://lists.openembedded.org/g/openembedded-core/message/191981

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/103045095/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#18 lore raw
Quoted 10 lines · Tim Orling via lists.openembedded.org
Quoted 5 lines · Alexandre Belloni

Hello Tim,

I got this failure, can you check?

Quoted 7 lines
Quoted 5 lines

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6166/steps/14/logs/stdio

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6159/steps/14/logs/stdio

this is from Peter’s change which had not been merged when I sent this

series.

Correction: it is from Julien’s pypi recipetool change. The pypi.bbclass does not create SRC_URI checksums. The inly test case other than mine with SRC_URI checksums was for —no-pypi setuptools3 and do we have zero coverage.

Quoted 115 lines
Quoted 98 lines
2023-12-08 22:03:18,822 - oe-selftest - INFO - FAIL:
recipetool.RecipetoolCreateTests.test_recipetool_create_python3_pep517_maturin
(subunit.RemotedTestCase)
2023-12-08 22:03:18,822 - oe-selftest - INFO -
----------------------------------------------------------------------
2023-12-08 22:03:18,823 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call
last):
File
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/recipetool.py",
line 698, in test_recipetool_create_python3_pep517_maturin
self._test_recipe_contents(recipefile, checkvars, inherits)
File
"/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py",
line 178, in _test_recipe_contents
self.assertEqual(missingvars, {}, 'Some expected variables not found
in recipe: %s' % checkvars)
File "/usr/lib64/python3.11/unittest/case.py", line 873, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib64/python3.11/unittest/case.py", line 1172, in
assertDictEqual
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib64/python3.11/unittest/case.py", line 703, in fail
raise self.failureException(msg)
AssertionError: {'SRC_URI': 'https://files.pythonhosted.or[586
chars]238'} != {}
Diff is 656 characters long. Set self.maxDiff to None to see it. : Some
expected variables not found in recipe: {'SRC_URI': '
https://files.pythonhosted.org/packages/64/26/cffb93fe9c6b5a91c497f37fae14a4b073ecbc47fc36a9979c7aa888b245/pydantic_core-${PV}.tar.gz',
'SRC_URI[md5sum]': '1eb13c211147496c1c9484ff7f8ac438', 'SRC_URI[sha1sum]':
'0803a731aa793f3eaf4d52f656d7300408ca0b36', 'SRC_URI[sha256sum]':
'6d30226dfc816dd0fdf120cae611dd2215117e4f9b124af8c60ab9093b6e8e71',
'SRC_URI[sha384sum]':
'cca6d9f0ba72ff92e16aaf3d6c628b8df33bdf244cc39c9d59cff9d541f35d4e87ec136e60e4b040ad73c5c69b27cec6', 'SRC_URI[sha512sum]':
'e6c3081dfa58d9b59f8c33dfe80712d732ac3048a2f8240f9a0f5448d9bfc83e222c1e08b8bb49779b84f685c3bec3060cc935c113a818b1ab8aa955f3df2238'}

Quoted 11 lines · Tim Orling

We are increasingly seeing Rust extensions for Python added in the ecosystem. The current best practices approach has moved from using 'setuptools-rust' to using 'maturin'.

* Add python3-maturin, including 'native' and 'nativesdk' support.
* Add a 'python_maturin' bbclass for the PEP-517 backend.
* Add recipetool support for the 'maturin' PEP-517 backend.
* Add a very basic 'maturin list-python' smoke test added to oeqa for both 'runtime' (-c testimage) and 'sdk' (-c testsdk).

The following changes since commit

70ad9b9b309575134834757468cb32c9e3b87448:

Quoted 2 lines

bitbake: hashserv: sqlite: Ensure sync propagates to database

connections (2023-12-04 22:36:09 +0000)

Quoted 14 lines

are available in the Git repository at:

https://git.yoctoproject.org/poky-contrib timo/maturin_1.3.2
https://git.yoctoproject.org/poky-contrib/log/?h=timo/maturin_1.3.2

Tim Orling (6):
python3-setuptools-rust: BBCLASSEXTEND + nativesdk
python3-maturin: add v1.3.2
classes-recipe: add python_maturin.bbclass
recipetool: add python_maturin support
oe-selfest: add maturn runtime (testimage) test
python3-maturin: add sdk (testsdk) test case

Quoted 14 lines
 meta/classes-recipe/python_maturin.bbclass    |  17 +
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/lib/oeqa/runtime/cases/maturin.py        |  23 +
 meta/lib/oeqa/sdk/cases/maturin.py            |  33 +
 meta/lib/oeqa/selftest/cases/recipetool.py    |  34 ++
 .../python/python3-maturin-crates.inc         | 572 ++++++++++++++++++
 .../python/python3-maturin_1.3.2.bb           |  40 ++
 .../python/python3-setuptools-rust_1.8.1.bb   |   2 +-
 .../lib/recipetool/create_buildsys_python.py  |   1 +
 9 files changed, 722 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes-recipe/python_maturin.bbclass
 create mode 100644 meta/lib/oeqa/runtime/cases/maturin.py
 create mode 100644 meta/lib/oeqa/sdk/cases/maturin.py
 create mode 100644

meta/recipes-devtools/python/python3-maturin-crates.inc

Quoted 1 line
 create mode 100644 meta/recipes-devtools/python/

python3-maturin_1.3.2.bb

Quoted 8 lines

--
2.34.1

Quoted 5 lines

--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#192089):
https://lists.openembedded.org/g/openembedded-core/message/192089
Mute This Topic: https://lists.openembedded.org/mt/103045095/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-