Python PEP-517: build wheels and install with pip
v2 cover Python PEP-517: build wheels and install with pip
Upstream Python is moving to pyproject.toml rather than setup.py and wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages with setup.py install is deprecated.
We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS (at a minimum 2 years). This seems to be a compelling enough reason to bring in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into PYTHON_SITEPACKAGES_DIR. This allows all other recipes to be installed with pip, as intended by upstream.
Three new classes are introduced:
(1) flit_core which builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2) setuptools_build_meta which builds PEP-517 compliant packages which use setuptools_build_meta as their build-backend.
(3) pip_install_wheel, which simply installs wheels built by other classes.
The original setuptools3.bbclass do_compile is modified to 'setup.py bdist_wheel' (from python3-wheel-native) rather than the deprecated 'setup.py build' mechanism.
The original setuptools3.bbclass do_install is modified to use the pip_install_wheel.bbclass to install wheels with pip rather than the deprecated 'setup.py install' mechanism.
For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend should instead use of the new PEP-517 classes. This function is still in progress.
During testing, recipes which use python3-setuptools-scm are failing to determine the version of the python package, For several recipes, the fix was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes are still going to fail for the short term, until the root cause can be determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at this time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than --target) and also passing --prefix so that scripts are (usually) installed in the proper location. pip install also 'helpfully' overwrites python3 intepreter with nativepython3 in #!, so add a fix to return the values to what is actually correct (similar to what is done in setuptools3.bbclass). Also several recipes still needed PIP_INSTALL_PACKAGE defined.
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit 41ae26ecab731f2fb6593df143476c94183be209:
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contrib timo/bootstrap-wheels
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inherit flit_core pip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inherit setuptools_build_meta
python3-attrs: inherit setuptools_build_meta
python3-git: inherit setuptools_build_meta
python3-pytest: inherit setuptools_build_meta
python3-setuptools-scm: inherit setuptools_build_meta
python3-zipp: inherit setuptools_build_meta
python3-iniconfig: inherit setuptools_build_meta
python3-py: inherit setuptools_build_meta
python3-pluggy: inherit setuptools_build_meta
python3-setuptools: inherit setuptools_base_meta setuptools3.bbclass: refactor for wheels
python3-more-itertools: set PIP_INSTALL_PACKAGE
meson: inherit setuptools_build_meta
python3-libarchive-d: set PIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; set PIP_INSTALL_PACKAGE
python3-magic: set PIP_INSTALL_PACKAGE
bmap-tools: set PIP_INSTALL_PACKAGE, BASEVER
asciidoc: set PIP_INSTALL_PACKAGE
gi-docgen: set PIP_INSTALL_PACKAGE
numpy: set PIP_INSTALL_PACKAGE
python3-dbusmock: set PIP_INSTALL_PACKAGE
python3-mako: inherit setuptools_build_meta
python3-packaging: inherit setuptools_build_meta
python3-nose: drop recipe
meta/classes/flit_core.bbclass | 16 +++
meta/classes/pip_install_wheel.bbclass | 37 +++++++
meta/classes/setuptools3.bbclass | 20 ++--
meta/classes/setuptools_build_meta.bbclass | 18 ++++
meta/conf/distro/include/maintainers.inc | 3 +-
meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +-
meta/recipes-devtools/python/python-nose.inc | 18 ----
.../python/python-subunit.inc | 14 ---
.../python/python3-attrs_21.4.0.bb | 2 +-
.../python/python3-dbusmock_0.25.0.bb | 1 +
.../python/python3-flit-core_3.6.0.bb | 41 +++++++
.../python/python3-git_3.1.26.bb | 2 +-
.../python/python3-iniconfig_1.1.1.bb | 4 +-
.../python/python3-libarchive-c_4.0.bb | 2 +
.../python/python3-magic_0.4.25.bb | 1 +
.../python/python3-mako_1.1.6.bb | 2 +-
.../python/python3-more-itertools_8.12.0.bb | 2 +
.../python/python3-nose_1.3.7.bb | 2 -
.../python/python3-numpy_1.22.2.bb | 1 +
.../python/python3-packaging_21.3.bb | 2 +-
.../python/python3-pip_22.0.3.bb | 19 +++-
.../python/python3-pluggy_1.0.0.bb | 2 +-
.../python/python3-py_1.11.0.bb | 2 +-
.../python/python3-pytest_7.0.1.bb | 4 +-
.../python/python3-scons-native_4.3.0.bb | 7 --
.../python/python3-scons_4.3.0.bb | 13 ++-
.../python/python3-setuptools-scm_6.4.2.bb | 2 +-
.../python/python3-setuptools_59.5.0.bb | 19 +++-
.../0001-Change-hash-bang-to-python3.patch | 47 ++++++++
.../python/python3-smartypants_2.0.0.bb | 1 +
...001-setup.py-use-vendored-_distutils.patch | 27 +++++
.../python/python3-strict-rfc3339_0.7.bb | 4 +-
.../python/python3-subunit_1.4.0.bb | 17 ++-
...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++
.../python/python3-wheel_0.37.1.bb | 24 +++++
.../python/python3-zipp_3.7.0.bb | 2 +-
.../asciidoc/asciidoc_10.1.1.bb | 1 +
meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 +
.../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +-
39 files changed, 412 insertions(+), 78 deletions(-)
create mode 100644 meta/classes/flit_core.bbclass
create mode 100644 meta/classes/pip_install_wheel.bbclass
create mode 100644 meta/classes/setuptools_build_meta.bbclass
delete mode 100644 meta/recipes-devtools/python/python-nose.inc
delete mode 100644 meta/recipes-devtools/python/python-subunit.inc
create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
create mode 100644 meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch
create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch
create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} (86%)Signature · 1 line
2.30.2
PATCH v2 1/32 python3-wheel: move 0.37.0 from meta-python
This is one of the "new build tools" which are part of pypa (Python Packaging Authority) toolchain.
Wheels are the official delivery mechanism for Python packages, replacing the now deprecated Eggs (egg-info).
[YOCTO #14638]
--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-devtools/python/python3-wheel_0.37.1.bb | 12 ++++++++++++ 2 files changed, 13 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -651,6 +651,7 @@ RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander Kanavin <alex.kanavin@gmail. RECIPE_MAINTAINER:pn-python3-vcversioner = "Bruce Ashfield <[email protected]>" RECIPE_MAINTAINER:pn-python3-wcwidth = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-webcolors = "Bruce Ashfield <[email protected]>" +RECIPE_MAINTAINER:pn-python3-wheel = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-zipp = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-qemu = "Richard Purdie <[email protected]>" RECIPE_MAINTAINER:pn-qemu-helper-native = "Richard Purdie <[email protected]>"
meta/recipes-devtools/python/python3-wheel_0.37.1.bb new file +12 -0
@@ -0,0 +1,12 @@ +SUMMARY = "The official binary distribution format for Python " +HOMEPAGE = "https://github.com/pypa/wheel" +SECTION = "devel/python" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=10;endline=10;md5=8227180126797a0148f94f483f3e1489" + +SRC_URI[sha256sum] = "e9a504e793efbca1b8e0e9cb979a249cf4a0a7b5b8c9e8b65a5e39d49529c1c4" + +inherit pypi setuptools3 + +BBCLASSEXTEND = "native" +
Signature · 1 line
2.30.2
Re: [OE-core] [PATCH v2 01/32] python3-wheel: move 0.37.0 from meta-python
this series is breaking things quite a bit in rest of world.
https://errors.yoctoproject.org/Errors/Build/141298/
Quoted 58 lines · Tim Orling
This is one of the "new build tools" which are part of pypa (Python Packaging Authority) toolchain.
Wheels are the official delivery mechanism for Python packages, replacing the now deprecated Eggs (egg-info).
[YOCTO #14638]
Signed-off-by Tim Orling <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-devtools/python/python3-wheel_0.37.1.bb | 12 ++++++++++++ 2 files changed, 13 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bbmeta/conf/distro/include/maintainers.inc +1
-0@@ -651,6 +651,7 @@ RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander Kanavin <alex.kanavin@gmail. RECIPE_MAINTAINER:pn-python3-vcversioner = "Bruce Ashfield <[email protected]>" RECIPE_MAINTAINER:pn-python3-wcwidth = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-webcolors = "Bruce Ashfield <[email protected]>" +RECIPE_MAINTAINER:pn-python3-wheel = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-zipp = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-qemu = "Richard Purdie <[email protected]>" RECIPE_MAINTAINER:pn-qemu-helper-native = "Richard Purdie <[email protected]>"meta/recipes-devtools/python/python3-wheel_0.37.1.bb new file +12
-1@@ -0,0 +1,12 @@ +SUMMARY = "The official binary distribution format for Python " +HOMEPAGE = "https://github.com/pypa/wheel" +SECTION = "devel/python" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=10;endline=10;md5=8227180126797a0148f94f483f3e1489" + +SRC_URI[sha256sum] = "e9a504e793efbca1b8e0e9cb979a249cf4a0a7b5b8c9e8b65a5e39d49529c1c4" + +inherit pypi setuptools3 + +BBCLASSEXTEND = "native" + --2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162077): https://lists.openembedded.org/g/openembedded-core/message/162077
Mute This Topic: https://lists.openembedded.org/mt/89304009/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 01/32] python3-wheel: move 0.37.0 from meta-python
Quoted 3 lines · Khem Raj
this series is breaking things quite a bit in rest of world.
I understand, but we do need to be able to modernize. And getting oe-core stable is currently consuming all bandwidth.
Most of the failures will be because the wheel has a name like setuptools_rust*.whl but PYPI_PACKAGE = 'setuptools-rust' The fix is to set PIP_INSTALL_PACKAGE = "setuptools_rust"
Quoted 2 lines
Quoted 73 lines · Tim Orling
Quoted 9 lines
This is one of the "new build tools" which are part of pypa (Python Packaging Authority) toolchain.
Wheels are the official delivery mechanism for Python packages, replacing the now deprecated Eggs (egg-info).
[YOCTO #14638]
Review
Quoted 8 lines
Signed-off-by Tim Orling <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 + meta/recipes-devtools/python/python3-wheel_0.37.1.bb | 12 ++++++++++++ 2 files changed, 13 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb(quoted diff) +0
-0b/meta/conf/distro/include/maintainers.inc
meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander
Quoted 4 lines
index
740e5fe55c4..72cd0125814100644meta/conf/distro/include/maintainers.inc +0
-0@@ -651,6 +651,7 @@ RECIPE_MAINTAINER:pn-python3-typogrify = "AlexanderKanavin <alex.kanavin@gmail.
meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 651
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-vcversioner = "Bruce Ashfield <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 652
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-wcwidth = "Tim Orling <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 653
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-webcolors = "Bruce Ashfield <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 654
Quoted 1 line
meta/conf/distro/include/maintainers.inc +1
-0+RECIPE_MAINTAINER:pn-python3-wheel = "Tim Orling <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 655
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-zipp = "Tim Orling <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 656
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-qemu = "Richard Purdie <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-typogrify = "Alexander · line 657
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-qemu-helper-native = "Richard Purdie <Review
Quoted 1 line
(quoted diff) +0
-0b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
meta/recipes-devtools/python/python3-wheel_0.37.1.bb line 5
Quoted 10 lines
new file mode 100644
index 00000000000..6c7a31db9e7meta/recipes-devtools/python/python3-wheel_0.37.1.bb new file +5
-0@@ -0,0 +1,12 @@ +SUMMARY = "The official binary distribution format for Python " +HOMEPAGE = "https://github.com/pypa/wheel" +SECTION = "devel/python" +LICENSE = "MIT" +LIC_FILES_CHKSUM ="file://PKG-INFO;beginline=10;endline=10;md5=8227180126797a0148f94f483f3e1489"
meta/recipes-devtools/python/python3-wheel_0.37.1.bb line 7
Quoted 2 lines
meta/recipes-devtools/python/python3-wheel_0.37.1.bb new file +2
-0+ +SRC_URI[sha256sum] ="
e9a504e793efbca1b8e0e9cb979a249cf4a0a7b5b8c9e8b65a5e39d49529c1c4"Quoted 9 lines
meta/recipes-devtools/python/python3-wheel_0.37.1.bb new file +5
-1+ +inherit pypi setuptools3 + +BBCLASSEXTEND = "native" + --2.30.2
Quoted 3 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162077):https://lists.openembedded.org/g/openembedded-core/message/162077
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/89304009/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
PATCH v2 2/32 python3-flit-core: add recipe for 3.6.0
In reply to Tim OrlingThis is the core of one of the "new build system" tools--hosted officially as part of the pypa (Python Packaging Authority) repositories--it is an increasingly common build tool (e.g. typing_extensions and tomli) as declared in pyproject.toml for said packages.
This package provides a very simple bootstrapping method that builds the source tarball (build_sdist) and a wheel (build_wheel).
Bootstrap -native by simply unzipping the wheel to PYTHON_SITEPACKAGES_DIR.
Use pip to install the wheel for class-target.
Wheels are the official vehicle for delivering Python packages now. Eggs (egg-info) are deprecated and will increasingly go away.
[YOCTO #14638]
--- meta/conf/distro/include/maintainers.inc | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++++++++++++++ 2 files changed, 42 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
meta/conf/distro/include/maintainers.inc +1 -0
@@ -599,6 +599,7 @@ RECIPE_MAINTAINER:pn-python3-pycryptodome = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER:pn-python3-pycryptodomex = "Joshua Watt <[email protected]>" RECIPE_MAINTAINER:pn-python3-pyrsistent = "Bruce Ashfield <[email protected]>" RECIPE_MAINTAINER:pn-python3-extras = "Oleksandr Kravchuk <[email protected]>" +RECIPE_MAINTAINER:pn-python3-flit-core = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-git = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER:pn-python3-gitdb = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER:pn-python3-hypothesis = "Tim Orling <[email protected]>"
meta/recipes-devtools/python/python3-flit-core_3.6.0.bb new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.30.2
PATCH v2 3/32 flit_core.bbclass: add helper for newer python packaging
In reply to Tim OrlingSome python packages now use pyproject.toml and declare flit_core.buildapi as the build engine
Use pip_install_wheel class to install.
[YOCTO #14638]
--- meta/classes/flit_core.bbclass | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 meta/classes/flit_core.bbclass
meta/classes/flit_core.bbclass new file +16 -0
@@ -0,0 +1,16 @@ +inherit pip_install_wheel python3native python3-dir + +DEPENDS += "python3 python3-flit-core-native python3-pip-native" + +do_configure () { + mkdir -p ${S}/dist + cat > ${S}/build-it.py << EOF +from flit_core import buildapi +buildapi.build_wheel('./dist') +EOF +} + +do_compile () { + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} ${S}/build-it.py +} +
Signature · 1 line
2.30.2
PATCH v2 4/32 python3-wheel: inherit flit_core
In reply to Tim OrlingBackport pyproject.toml from flit-backend branch. Inherit flit_core class to build
Inherit pip_install_wheel to install wheels for target
We need to bootstrap python3-wheel-native in order to have bdist_wheel available to python3-setuptools-native and the refactored setuptools3.bbclass. Simply unzip the wheel into PYTHON_SITEPACKAGES_DIR for class-native.
[YOCTO #14638]
--- ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++ .../python/python3-wheel_0.37.1.bb | 16 ++- 2 files changed, 114 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch new file +100 -0
@@ -0,0 +1,100 @@ +From f00dd220346773bc088d403847ee7f06f2b4c30a Mon Sep 17 00:00:00 2001 +From: Tim Orling <[email protected]> +Date: Fri, 18 Feb 2022 11:09:16 -0800 +Subject: [PATCH] Backport pyproject.toml from flit-backend branch + +This allows us to bootstrap wheels and PEP-517 packaging. + +Upstream-Status: Backport from flit-backend branch +https://raw.githubusercontent.com/pypa/wheel/4f6ba78fede38a8d9e35a14e38377a121033afb3/pyproject.toml + +Signed-off-by: Tim Orling <[email protected]> +--- + pyproject.toml | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 78 insertions(+) + create mode 100644 pyproject.toml + +diff --git a/pyproject.toml b/pyproject.toml +new file mode 100644 +index 0000000..749b8de +--- /dev/null ++++ b/pyproject.toml +@@ -0,0 +1,78 @@ ++[build-system] ++requires = ["flit_core >=3.2,<4"] ++build-backend = "flit_core.buildapi" ++ ++[project] ++name = "wheel" ++description = "A built-package format for Python" ++readme = "README.rst" ++classifiers = [ ++ "Development Status :: 5 - Production/Stable", ++ "Intended Audience :: Developers", ++ "Topic :: System :: Archiving :: Packaging", ++ "License :: OSI Approved :: MIT License", ++ "Programming Language :: Python", ++ "Programming Language :: Python :: 3 :: Only", ++ "Programming Language :: Python :: 3.7", ++ "Programming Language :: Python :: 3.8", ++ "Programming Language :: Python :: 3.9", ++ "Programming Language :: Python :: 3.10" ++] ++authors = [{name = "Daniel Holth", email = "[email protected]"}] ++maintainers = [{name = "Alex Grönholm", email = "[email protected]"}] ++keywords = ["wheel", "packaging"] ++license = {file = "LICENSE.txt"} ++requires-python = ">=3.7" ++dependencies = [ ++ "setuptools >= 45.2.0" ++] ++dynamic = ["version"] ++ ++[project.urls] ++Documentation = "https://wheel.readthedocs.io/" ++Changelog = "https://wheel.readthedocs.io/en/stable/news.html" ++"Issue Tracker" = "https://github.com/pypa/wheel/issues" ++ ++[project.scripts] ++wheel = "wheel.cli:main" ++ ++[project.entry-points."distutils.commands"] ++bdist_wheel = "wheel.bdist_wheel:bdist_wheel" ++ ++[project.optional-dependencies] ++test = [ ++ "pytest >= 3.0.0" ++] ++ ++[tool.flit.sdist] ++exclude = [ ++ ".cirrus.yml", ++ ".github/*", ++ ".gitignore", ++ ".pre-commit-config.yaml", ++ ".readthedocs.yml" ++] ++ ++[tool.black] ++target-version = ['py37'] ++extend-exclude = ''' ++^/src/wheel/vendored/ ++''' ++ ++[tool.isort] ++src_paths = ["src"] ++profile = "black" ++skip_gitignore = true ++ ++[tool.flake8] ++max-line-length = 88 ++ ++[tool.pytest.ini_options] ++testpaths = "tests" ++ ++[tool.coverage.run] ++source = ["wheel"] ++omit = ["*/vendored/*"] ++ ++[tool.coverage.report] ++show_missing = true
meta/recipes-devtools/python/python3-wheel_0.37.1.bb +14 -2
@@ -6,7 +6,19 @@ LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=10;endline=10;md5=8227180126797a01 SRC_URI[sha256sum] = "e9a504e793efbca1b8e0e9cb979a249cf4a0a7b5b8c9e8b65a5e39d49529c1c4" -inherit pypi setuptools3 +inherit flit_core pypi setuptools3-base -BBCLASSEXTEND = "native" +SRC_URI += " file://0001-Backport-pyproject.toml-from-flit-backend-branch.patch" + +DEPENDS:remove:class-native = "python3-pip-native" + +do_install:class-native () { + # We need to bootstrap python3-wheel-native + install -d ${D}${PYTHON_SITEPACKAGES_DIR} + PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ + bbfatal_log "Failed to install" +} + +BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.30.2
PATCH v2 5/32 pip_install_wheel.bbclass: add helper class
In reply to Tim OrlingProvide a helper class to use pip to install wheels built by either bdist_wheel or a PEP-517 backend.
Set pip install arguments via PIP_INSTALL_ARGS, which can be overriden by recipes.
Pass --root and --prefix to ensure that pip installs things into the proper place in sysroot.
By passing --no-deps and --no-index we avoid finicky dependency checking (pip expects wheels in its cache) and avoid trying to fetch wheels from pypi.org. This is basically the same behavior we have now, the dependencies should be declared in the recipe.
pip install changes the python interpreter in scripts installed in ${bindir}, e.g. to #!/usr/bin/nativepython3, correct the behavior after install to #!/usr/bin/env python3.
[YOCTO #14638]
--- meta/classes/pip_install_wheel.bbclass | 37 ++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 meta/classes/pip_install_wheel.bbclass
meta/classes/pip_install_wheel.bbclass new file +37 -0
@@ -0,0 +1,37 @@ +DEPENDS:append = " python3-pip-native" + +PIP_INSTALL_PACKAGE ?= "${PYPI_PACKAGE}" +PIP_INSTALL_DIST_PATH ?= "${B}/dist" +PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl" + +PIP_INSTALL_ARGS ?= "\ + -vvvv \ + --no-deps \ + --no-index \ + --root=${D} \ + --prefix=${prefix} \ +" + +pip_install_wheel_do_install:prepend () { + install -d ${D}${PYTHON_SITEPACKAGES_DIR} +} + +export PYPA_WHEEL + +PIP_INSTALL_PYTHON = "python3" +PIP_INSTALL_PYTHON:class-native = "nativepython3" + +pip_install_wheel_do_install () { + nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} || + bbfatal_log "Failed to pip install wheel. Check the logs." + + for i in ${D}${bindir}/* ${D}${sbindir}/*; do + if [ -f "$i" ]; then + sed -i -e "1s,#!.*nativepython3,#!${USRBINPATH}/env ${PIP_INSTALL_PYTHON}," $i + sed -i -e "s:${PYTHON}:${USRBINPATH}/env\ ${PIP_INSTALL_PYTHON}:g" $i + sed -i -e "s:${STAGING_BINDIR_NATIVE}:${bindir}:g" $i + fi + done +} + +EXPORT_FUNCTIONS do_install
Signature · 1 line
2.30.2
PATCH v2 6/32 setuptools_build_meta.bbclass: add helper class
In reply to Tim OrlingThis class uses the PEP 517 compliant setuptools.buil_meta to build wheels.
[YOCTO #14638]
--- meta/classes/setuptools_build_meta.bbclass | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 meta/classes/setuptools_build_meta.bbclass
meta/classes/setuptools_build_meta.bbclass new file +18 -0
@@ -0,0 +1,18 @@ +inherit pip_install_wheel setuptools3-base + +DEPENDS += "python3 python3-setuptools-native python3-wheel-native" + +setuptools_build_meta_do_configure () { + mkdir -p ${S}/dist + cat > ${S}/build-it.py << EOF +from setuptools import build_meta +wheel = build_meta.build_wheel('./dist') +print(wheel) +EOF +} + +setuptools_build_meta_do_compile () { + nativepython3 ${S}/build-it.py +} + +EXPORT_FUNCTIONS do_configure do_compile
Signature · 1 line
2.30.2
PATCH v2 7/32 python3-pip: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares setuptools.build_meta as the backend for PEP-517 packaging.
We bootstrap the -native installation by simply unzipping the wheel to PYTHON_SITEPACKAGES_DIR, so that all other recipes can use pip to install wheels (as intended by upstream).
[YOCTO #14638]
--- .../python/python3-pip_22.0.3.bb | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
meta/recipes-devtools/python/python3-pip_22.0.3.bb +17 -2
@@ -4,17 +4,32 @@ SECTION = "devel/python" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=63ec52baf95163b597008bb46db68030" +inherit pypi setuptools_build_meta + DEPENDS += "python3 python3-setuptools-native" -inherit pypi setuptools3 +# To avoid a dependency loop; we bootstrap -native +DEPENDS:remove:class-native = "python3-pip-native" +DEPENDS:append:class-native = " unzip-native" SRC_URI += "file://0001-change-shebang-to-python3.patch" SRC_URI[sha256sum] = "f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0" +PYPA_WHEEL ?= "${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" + +do_install:class-native() { + # Bootstrap to prevent dependency loop in python3-pip-native + install -d ${D}${PYTHON_SITEPACKAGES_DIR} + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ + bbfatal_log "Failed to unzip wheel: ${PYPA_WHEEL}. Check the logs." +} + do_install:append() { # Install as pip3 and leave pip2 as default - rm ${D}/${bindir}/pip + if [ -f ${D}/${bindir}/pip ]; then + rm ${D}/${bindir}/pip + fi } RDEPENDS:${PN} = "\
Signature · 1 line
2.30.2
PATCH v2 8/32 python3-attrs: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-attrs_21.4.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-attrs_21.4.0.bb +1 -1
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=d4ab25949a73fe7d4fdee93bcbdbf8ff" SRC_URI[sha256sum] = "626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta RDEPENDS:${PN}:class-target += " \ ${PYTHON_PN}-crypt \
Signature · 1 line
2.30.2
PATCH v2 9/32 python3-git: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-git_3.1.26.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-git_3.1.26.bb +1 -1
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=8b8d26c37c1d5a04f9b0186edbebc183" PYPI_PACKAGE = "GitPython" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta SRC_URI[sha256sum] = "fc8868f63a2e6d268fb25f481995ba185a85a66fcad126f039323ff6635669ee"
Signature · 1 line
2.30.2
PATCH v2 10/32 python3-pytest: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-pytest_7.0.1.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-devtools/python/python3-pytest_7.0.1.bb +2 -2
@@ -7,9 +7,9 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=bd27e41b6550fe0fc45356d1d81ee37c" SRC_URI[sha256sum] = "e30905a0c131d3d94b89624a1cc5afec3e0ba2fbdb151867d8e0ebd49850f171" -DEPENDS = "python3-setuptools-scm-native" +DEPENDS += "python3-setuptools-scm-native" -inherit update-alternatives pypi setuptools3 +inherit update-alternatives pypi setuptools_build_meta RDEPENDS:${PN}:class-target += " \ ${PYTHON_PN}-atomicwrites \
Signature · 1 line
2.30.2
PATCH v2 11/32 python3-setuptools-scm: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb +1 -1
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=8227180126797a0148 SRC_URI[sha256sum] = "6833ac65c6ed9711a4d5d2266f8024cfa07c533a0e55f4c12f6eff280a5a9e30" PYPI_PACKAGE = "setuptools_scm" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta UPSTREAM_CHECK_REGEX = "setuptools_scm-(?P<pver>.*)\.tar"
Signature · 1 line
2.30.2
PATCH v2 12/32 python3-zipp: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-zipp_3.7.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-zipp_3.7.0.bb +1 -1
@@ -7,7 +7,7 @@ SRC_URI[sha256sum] = "9f50f446828eb9d45b267433fd3e9da8d801f614129124863f9c51ebce DEPENDS += "${PYTHON_PN}-setuptools-scm-native" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta DEPENDS += "${PYTHON_PN}-toml-native"
Signature · 1 line
2.30.2
PATCH v2 13/32 python3-iniconfig: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which decalres the setuptools.build_meta backend for PEP-517 packaging.
DEPENDS on python3-setuptools-scm-native (as declared in the pyproject.toml build-system.requires).
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb +3 -1
@@ -7,4 +7,6 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=a6bb0320b04a0a503f12f69fea479de9" SRC_URI[md5sum] = "0b7f3be87481211c183eae095bcea6f1" SRC_URI[sha256sum] = "bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32" -inherit pypi setuptools3 +DEPENDS += "python3-setuptools-scm-native" + +inherit pypi setuptools_build_meta
Signature · 1 line
2.30.2
PATCH v2 14/32 python3-py: inherit setuptools_build_meta
In reply to Tim OrlingUpstreama provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-py_1.11.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-py_1.11.0.bb +1 -1
@@ -7,7 +7,7 @@ SRC_URI[sha256sum] = "51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4 DEPENDS += "${PYTHON_PN}-setuptools-scm-native" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.30.2
PATCH v2 15/32 python3-pluggy: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which while it does not declare the setuptools.build_meta backend is compatible with it.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-pluggy_1.0.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-pluggy_1.0.0.bb +1 -1
@@ -10,7 +10,7 @@ RDEPENDS:${PN} += "${PYTHON_PN}-importlib-metadata \ ${PYTHON_PN}-more-itertools \ " -inherit pypi ptest setuptools3 +inherit pypi ptest setuptools_build_meta SRC_URI += " \ file://run-ptest \
Signature · 1 line
2.30.2
PATCH v2 16/32 python3-setuptools: inherit setuptools_base_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
We need to bootstrap python3-setuptools-native, simply installing by unzipping the built wheel. This avoids a dependency loop.
[YOCTO #14638]
--- .../python/python3-setuptools_59.5.0.bb | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
meta/recipes-devtools/python/python3-setuptools_59.5.0.bb +14 -5
@@ -4,7 +4,7 @@ SECTION = "devel/python" LICENSE = "MIT" LIC_FILES_CHKSUM = "file://LICENSE;beginline=1;endline=19;md5=7a7126e068206290f3fe9f8d6c713ea6" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta SRC_URI:append:class-native = " file://0001-conditionally-do-not-fetch-code-by-easy_install.patch"
@@ -17,6 +17,19 @@ SRC_URI[sha256sum] = "d144f85102f999444d06f9c0e8c737fd0194f10f2f7e5fdb77573f6e2f DEPENDS += "${PYTHON_PN}" +# Avoid dependency loop; we bootstrap -native +DEPENDS:remove:class-native = "python3-pip-native python3-setuptools-native" +DEPENDS:append:class-native = " unzip-native" + +PYPA_WHEEL ?= "${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" + +do_install:class-native() { + # Bootstrap to prevent dependency loop in python3-pip-native + install -d ${D}${PYTHON_SITEPACKAGES_DIR} + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ + bbfatal_log "Failed to unzip wheel: ${PYPA_WHEEL}. Check the logs." +} + RDEPENDS:${PN} = "\ ${PYTHON_PN}-2to3 \ ${PYTHON_PN}-compile \
@@ -38,10 +51,6 @@ RDEPENDS:${PN} = "\ ${PYTHON_PN}-xml \ " -do_install:prepend() { - install -d ${D}${PYTHON_SITEPACKAGES_DIR} -} - BBCLASSEXTEND = "native nativesdk" # The pkg-resources module can be used by itself, without the package downloader
Signature · 1 line
2.30.2
PATCH v2 17/32 setuptools3.bbclass: refactor for wheels
In reply to Tim OrlingDepend on python3-wheel-native so that we can build with 'setup.py bdist_wheel'.
Use pip_install_wheel class to install the built wheels with pip, as intended by upstream Python.
[YOCTO #14638]
--- meta/classes/setuptools3.bbclass | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-)
meta/classes/setuptools3.bbclass +8 -12
@@ -1,6 +1,7 @@ -inherit setuptools3-base +inherit setuptools3-base pip_install_wheel -B = "${WORKDIR}/build" +# bdist_wheel builds in ./dist +#B = "${WORKDIR}/build" SETUPTOOLS_BUILD_ARGS ?= "" SETUPTOOLS_INSTALL_ARGS ?= "--root=${D} \
@@ -23,20 +24,15 @@ setuptools3_do_compile() { STAGING_INCDIR=${STAGING_INCDIR} \ STAGING_LIBDIR=${STAGING_LIBDIR} \ ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \ - build --build-base=${B} ${SETUPTOOLS_BUILD_ARGS} || \ - bbfatal_log "'${PYTHON_PN} setup.py build ${SETUPTOOLS_BUILD_ARGS}' execution failed." + bdist_wheel ${SETUPTOOLS_BUILD_ARGS} || \ + bbfatal_log "'${PYTHON_PN} setup.py bdist_wheel ${SETUPTOOLS_BUILD_ARGS}' execution failed." } setuptools3_do_compile[vardepsexclude] = "MACHINE" setuptools3_do_install() { cd ${SETUPTOOLS_SETUP_PATH} - install -d ${D}${PYTHON_SITEPACKAGES_DIR} - STAGING_INCDIR=${STAGING_INCDIR} \ - STAGING_LIBDIR=${STAGING_LIBDIR} \ - PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \ - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \ - build --build-base=${B} install --skip-build ${SETUPTOOLS_INSTALL_ARGS} || \ - bbfatal_log "'${PYTHON_PN} setup.py install ${SETUPTOOLS_INSTALL_ARGS}' execution failed." + + pip_install_wheel_do_install # support filenames with *spaces* find ${D} -name "*.py" -exec grep -q ${D} {} \; \
@@ -64,5 +60,5 @@ setuptools3_do_install[vardepsexclude] = "MACHINE" EXPORT_FUNCTIONS do_configure do_compile do_install export LDSHARED="${CCLD} -shared" -DEPENDS += "python3-setuptools-native" +DEPENDS += "python3-setuptools-native python3-wheel-native"
Signature · 1 line
2.30.2
PATCH v2 18/32 python3-more-itertools: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingDefine PIP_INSTALL_PACKAGE as more_itertools to match the name of the wheel.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb +2 -0
@@ -7,6 +7,8 @@ SRC_URI[sha256sum] = "7dc6ad46f05f545f900dd59e8dfb4e84a4827b97b3cfecb175ea0c7d24 inherit pypi setuptools3 ptest +PIP_INSTALL_PACKAGE = "more_itertools" + SRC_URI += " \ file://run-ptest \ "
Signature · 1 line
2.30.2
PATCH v2 19/32 meson: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares setuptools.build_meta to be backend for PEP-517 packaging.
Set PIP_INSTALL_PACKAGE as this recipe does not inherit pypi and therefor no PYPI_PACKAGE is defined
[YOCTO #14638]
--- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/recipes-devtools/meson/meson_0.61.2.bb +3 -1
@@ -21,7 +21,9 @@ SRC_URI[sha256sum] = "0233a7f8d959079318f6052b0939c27f68a5de86ba601f25c9ee6869fb UPSTREAM_CHECK_URI = "https://github.com/mesonbuild/meson/releases" UPSTREAM_CHECK_REGEX = "meson-(?P<pver>\d+(\.\d+)+)\.tar" -inherit setuptools3 +inherit setuptools_build_meta + +PIP_INSTALL_PACKAGE = "meson" RDEPENDS:${PN} = "ninja python3-modules python3-pkg-resources"
Signature · 1 line
2.30.2
Re: [OE-core] [PATCH v2 19/32] meson: inherit setuptools_build_meta
Hi,
Could you look into this build https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/4103/steps/15/logs/stdio
Error:
Problem: package nativesdk-packagegroup-sdk-host-1.0-r12.aarch64_nativesdk requires nativesdk-meson, but none of the providers can be installed
- conflicting requests
- nothing provides nativesdk-python3-pkg-resources needed by nativesdk-meson-0.61.2-r0.aarch64_nativesdk
(try to add '--skip-broken' to skip uninstallable packages)
Regards,
Oleksiy
________________________________________
From: [email protected] <[email protected]> on behalf of Tim Orling <[email protected]>
Sent: Monday, February 21, 2022 23:23
To: [email protected]
Subject: [OE-core] [PATCH v2 19/32] meson: inherit setuptools_build_meta
Upstream provides a pyproject.toml which declares setuptools.build_meta to be backend for PEP-517 packaging.
Set PIP_INSTALL_PACKAGE as this recipe does not inherit pypi and therefor no PYPI_PACKAGE is defined
[YOCTO #14638]
--- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/recipes-devtools/meson/meson_0.61.2.bb +3 -1
@@ -21,7 +21,9 @@ SRC_URI[sha256sum] = "0233a7f8d959079318f6052b0939c27f68a5de86ba601f25c9ee6869fb UPSTREAM_CHECK_URI = "https://github.com/mesonbuild/meson/releases" UPSTREAM_CHECK_REGEX = "meson-(?P<pver>\d+(\.\d+)+)\.tar" -inherit setuptools3 +inherit setuptools_build_meta + +PIP_INSTALL_PACKAGE = "meson" RDEPENDS:${PN} = "ninja python3-modules python3-pkg-resources"
--
2.30.2
PATCH v2 20/32 python3-libarchive-d: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingSet PIP_INSTALL_PACKAGE to libarchive_c to match the name of the wheel built by bdist_wheel.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-libarchive-c_4.0.bb | 2 ++ 1 file changed, 2 insertions(+)
meta/recipes-devtools/python/python3-libarchive-c_4.0.bb +2 -0
@@ -9,6 +9,8 @@ PYPI_PACKAGE = "libarchive-c" inherit pypi setuptools3 +PIP_INSTALL_PACKAGE = "libarchive_c" + SRC_URI[sha256sum] = "a5b41ade94ba58b198d778e68000f6b7de41da768de7140c984f71d7fa8416e5" RDEPENDS:${PN} += "\
Signature · 1 line
2.30.2
PATCH v2 21/32 python3-smartypants: patch hash bang to python3
In reply to Tim OrlingChange hash bang in smartypants and smartypants.py to be /usr/bin/env python3
[YOCTO #14638]
--- .../0001-Change-hash-bang-to-python3.patch | 47 +++++++++++++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + 2 files changed, 48 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch new file +47 -0
@@ -0,0 +1,47 @@ +From e299bcb05405ff49450307bf3002c1fac14a866c Mon Sep 17 00:00:00 2001 +From: Tim Orling <[email protected]> +Date: Sun, 20 Feb 2022 18:55:50 -0800 +Subject: [PATCH] Change hash bang to python3 + +In setup.py, smartypants and smartypants.py set hash bang to python3. + +Upstream-Status: Inappropriate [oe-core specific] + +Signed-off-by: Tim Orling <[email protected]> + +--- + setup.py | 2 +- + smartypants | 2 +- + smartypants.py | 2 +- + 3 files changed, 3 insertions(+), 3 deletions(-) + +diff --git a/setup.py b/setup.py +index ff1ea76..96a8b73 100755 +--- a/setup.py ++++ b/setup.py +@@ -1,4 +1,4 @@ +-#!/usr/bin/env python ++#!/usr/bin/env python3 + # Copyright (C) 2013, 2014 by Yu-Jie Lin + # For detail license information, See COPYING + +diff --git a/smartypants b/smartypants +index 189adf5..0cca568 100755 +--- a/smartypants ++++ b/smartypants +@@ -1,4 +1,4 @@ +-#!/usr/bin/env python ++#!/usr/bin/env python3 + # Copyright (c) 2013, 2014 Yu-Jie Lin + # Licensed under the BSD License, for detailed license information, see COPYING + +diff --git a/smartypants.py b/smartypants.py +index a70575b..e53bd87 100755 +--- a/smartypants.py ++++ b/smartypants.py +@@ -1,4 +1,4 @@ +-#!/usr/bin/python ++#!/usr/bin/env python3 + # -*- coding: utf-8 -*- + # Copyright (c) 2013, 2014, 2016 Yu-Jie Lin + # Copyright (c) 2004, 2005, 2007, 2013 Chad Miller
meta/recipes-devtools/python/python3-smartypants_2.0.0.bb +1 -0
@@ -6,6 +6,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=977036977591ac666c728921ecc54c4f" inherit pypi setuptools3 PYPI_PACKAGE = "smartypants" +SRC_URI += "file://0001-Change-hash-bang-to-python3.patch" SRC_URI[sha256sum] = "7812353a32022699a1aa8cd5626e01c94a946dcaeedaee2d0b382bae4c4cbf36" BBCLASSEXTEND = "native"
Signature · 1 line
2.30.2
PATCH v2 22/32 python3-scons: merge -native recipe
In reply to Tim OrlingThe native recipe simply calls create_wrapper to add a host script. Do this via do_install:append:class-native()
The scons*.1 man pages are being installed in ${prefix}, move them to ${mandir} (previously installed in ${datadir}).
[YOCTO #14638]
--- .../python/python3-scons-native_4.3.0.bb | 7 ------- meta/recipes-devtools/python/python3-scons_4.3.0.bb | 13 ++++++++++++- 2 files changed, 12 insertions(+), 8 deletions(-) delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
meta/recipes-devtools/python/python3-scons-native_4.3.0.bb deleted +0 -7
@@ -1,7 +0,0 @@ -require python3-scons_${PV}.bb -inherit native python3native -DEPENDS = "python3-native python3-setuptools-native" - -do_install:append() { - create_wrapper ${D}${bindir}/scons SCONS_LIB_DIR='${STAGING_DIR_HOST}/${PYTHON_SITEPACKAGES_DIR}' PYTHONNOUSERSITE='1' -}
meta/recipes-devtools/python/python3-scons_4.3.0.bb +12 -1
@@ -8,6 +8,7 @@ SRC_URI += " file://0001-Fix-man-page-installation.patch" SRC_URI[sha256sum] = "d47081587e3675cc168f1f54f0d74a69b328a2fc90ec4feb85f728677419b879" PYPI_PACKAGE = "SCons" +PIP_INSTALL_DIST_PATH = "${B}/build/dist" inherit pypi setuptools3
@@ -24,4 +25,14 @@ RDEPENDS:${PN}:class-target = "\ python3-pprint \ " -FILES:${PN}-doc += "${datadir}/scons*.1" +do_install:append() { + install -d ${D}${mandir}/man1 + mv ${D}${prefix}/scons*.1 ${D}${mandir}/man1/ +} +FILES:${PN}-doc += "${mandir}/man1/scons*.1" + +do_install:append:class-native() { + create_wrapper ${D}${bindir}/scons SCONS_LIB_DIR='${STAGING_DIR_HOST}/${PYTHON_SITEPACKAGES_DIR}' PYTHONNOUSERSITE='1' +} + +BBCLASSEXTEND = "native"
Signature · 1 line
2.30.2
PATCH v2 23/32 python3-subunit: merge inc; set PIP_INSTALL_PACKAGE
In reply to Tim OrlingWe have not supported python2 for some time now, so there is no reason to maintain a separate python-subunit.inc file.
The wheel built by bdist_wheel is named 'python_subunit-*.whl', so set PIP_INSALL_PACKAGE appropriately.
Add patch to use vendored _distutils from setuptools.
[YOCTO #14638]
--- .../python/python-subunit.inc | 14 ---------- ...001-setup.py-use-vendored-_distutils.patch | 27 +++++++++++++++++++ .../python/python3-strict-rfc3339_0.7.bb | 4 ++- .../python/python3-subunit_1.4.0.bb | 17 ++++++++++-- 4 files changed, 45 insertions(+), 17 deletions(-) delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch
meta/recipes-devtools/python/python-subunit.inc deleted +0 -14
@@ -1,14 +0,0 @@ -SUMMARY = "Python implementation of subunit test streaming protocol" -HOMEPAGE = "https://pypi.org/project/python-subunit/" -SECTION = "devel/python" -LICENSE = "Apache-2.0" -LIC_FILES_CHKSUM = "file://README.rst;beginline=1;endline=20;md5=909c08e291647fd985fbe5d9836d51b6" - -PYPI_PACKAGE = "python-subunit" - -SRC_URI[md5sum] = "30f1ab20651d94442dd9a7f8c9e8d633" -SRC_URI[sha256sum] = "042039928120fbf392e8c983d60f3d8ae1b88f90a9f8fd7188ddd9c26cad1e48" - -inherit pypi - -BBCLASSEXTEND = "nativesdk"
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch new file +27 -0
@@ -0,0 +1,27 @@ +From 857719e82daea0d85b734cac34cf569050724068 Mon Sep 17 00:00:00 2001 +From: Tim Orling <[email protected]> +Date: Sun, 20 Feb 2022 20:26:51 -0800 +Subject: [PATCH] setup.py: use vendored _distutils + +Deprecation warning of distutils is interferring with bdist_wheel build. + +For now, use the vendored setuptools._distutils.core. + +Upstream-Status: Pending [upstream appears unmaintained] + +Signed-off-by: Tim Orling <[email protected]> +--- + setup.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/setup.py b/setup.py +index 1bf87cf..a49fb8e 100644 +--- a/setup.py ++++ b/setup.py +@@ -1,5 +1,5 @@ + import os.path +-from distutils.core import setup ++from setuptools._distutils.core import setup + + readme_file = os.path.join(os.path.dirname(__file__), 'README.md') + readme = open(readme_file).read()
meta/recipes-devtools/python/python3-strict-rfc3339_0.7.bb +3 -1
@@ -3,9 +3,11 @@ HOMEPAGE = "https://pypi.org/project/strict-rfc3339/" LICENSE = "GPL-3.0-only" LIC_FILES_CHKSUM = "file://LICENSE;md5=8f0e2cd40e05189ec81232da84bd6e1a" -SRC_URI[md5sum] = "4d9b635b4df885bc37bc1189d66c9abc" +SRC_URI += "file://0001-setup.py-use-vendored-_distutils.patch" SRC_URI[sha256sum] = "5cad17bedfc3af57b399db0fed32771f18fc54bbd917e85546088607ac5e1277" inherit pypi setuptools3 +PIP_INSTALL_PACKAGE = "strict_rfc3339" + BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/python/python3-subunit_1.4.0.bb +15 -2
@@ -1,4 +1,17 @@ -inherit setuptools3 -require python-subunit.inc +SUMMARY = "Python implementation of subunit test streaming protocol" +HOMEPAGE = "https://pypi.org/project/python-subunit/" +SECTION = "devel/python" +LICENSE = "Apache-2.0" +LIC_FILES_CHKSUM = "file://README.rst;beginline=1;endline=20;md5=909c08e291647fd985fbe5d9836d51b6" + +PYPI_PACKAGE = "python-subunit" + +SRC_URI[sha256sum] = "042039928120fbf392e8c983d60f3d8ae1b88f90a9f8fd7188ddd9c26cad1e48" + +inherit pypi setuptools3 + +PIP_INSTALL_PACKAGE = "python_subunit" RDEPENDS:${PN} = " python3-testtools" + +BBCLASSEXTEND = "nativesdk"
Signature · 1 line
2.30.2
PATCH v2 24/32 python3-magic: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingThe wheel built by bdist_wheel has the name "python_magic-*.whl" so set PIP_INSTALL_PACKAGE accordingly.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-magic_0.4.25.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/python/python3-magic_0.4.25.bb +1 -0
@@ -8,6 +8,7 @@ LICENSE = "MIT" LIC_FILES_CHKSUM = "file://LICENSE;md5=61495c152d794e6be5799a9edca149e3" PYPI_PACKAGE = "python-magic" +PIP_INSTALL_PACKAGE = "python_magic" inherit pypi setuptools3
Signature · 1 line
2.30.2
PATCH v2 25/32 bmap-tools: set PIP_INSTALL_PACKAGE, BASEVER
In reply to Tim OrlingChange filename to _git.bb and use BASEVER to set PV.
The wheel built by bdist_wheel is named "bmap_tool-*.whl" set the PIP_INSTALL_PACKAGE name accordingly.
Use BASEVER to set PYPA_WHEEL to match the version in the wheel produced by bdist_wheel.
[YOCTO #14638]
---
.../bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} (86%)meta/recipes-support/bmap-tools/bmap-tools_git.bb renamed from meta/recipes-support/bmap-tools/bmap-tools_3.6.bb +5 -1
@@ -13,7 +13,8 @@ SRC_URI = "git://github.com/intel/${BPN};branch=master;protocol=https" SRCREV = "c0673962a8ec1624b5189dc1d24f33fe4f06785a" S = "${WORKDIR}/git" -PV .= "+git${SRCPV}" +BASEVER = "3.6" +PV = "${BASEVER}+git${SRCPV}" UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
@@ -23,4 +24,7 @@ RDEPENDS:${PN} = "python3-core python3-compression python3-mmap python3-setuptoo inherit python3native inherit setuptools3 +PIP_INSTALL_PACKAGE = "bmap_tools" +PYPA_WHEEL = "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${BASEVER}-*.whl" + BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.30.2
PATCH v2 26/32 asciidoc: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingThis recipe does not inherit pypi and therefor we need to manually set PIP_INSTALL_PACKAGE to reflect the wheel built by bdist_wheel.
[YOCTO #14638]
--- meta/recipes-extended/asciidoc/asciidoc_10.1.1.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-extended/asciidoc/asciidoc_10.1.1.bb +1 -0
@@ -20,6 +20,7 @@ S = "${WORKDIR}/git" export SGML_CATALOG_FILES="file://${STAGING_ETCDIR_NATIVE}/xml/catalog" inherit setuptools3 +PIP_INSTALL_PACKAGE = "asciidoc" CLEANBROKEN = "1" BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.30.2
PATCH v2 27/32 gi-docgen: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingThe name of the wheel as built by bdist_wheel is "gi_docgen", set PIP_INSTALL_PACKAGE appropriately.
[YOCTO #14638]
--- meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-gnome/gi-docgen/gi-docgen_git.bb +1 -0
@@ -16,6 +16,7 @@ SRCREV = "37b04455ff58cb2ec3f58917d0737c435344f2fb" S = "${WORKDIR}/git" inherit setuptools3 +PIP_INSTALL_PACKAGE = "gi_docgen" RDEPENDS:${PN} += "python3-asyncio python3-core python3-jinja2 python3-json python3-markdown python3-markupsafe python3-pygments python3-toml python3-typogrify python3-xml"
Signature · 1 line
2.30.2
PATCH v2 28/32 numpy: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingThe recipe does not inherit pypi so we need to manually set PIP_INSTALL_PACKAGE appropriately.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-numpy_1.22.2.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/python/python3-numpy_1.22.2.bb +1 -0
@@ -20,6 +20,7 @@ UPSTREAM_CHECK_REGEX = "(?P<pver>\d+(\.\d+)+)\.tar" DEPENDS += "python3-cython-native" inherit ptest setuptools3 +PIP_INSTALL_PACKAGE = "numpy" S = "${WORKDIR}/numpy-${PV}"
Signature · 1 line
2.30.2
PATCH v2 29/32 python3-dbusmock: set PIP_INSTALL_PACKAGE
In reply to Tim OrlingThe name of the wheel as built by bdist_wheel is "python_dbusmock", set PIP_INSTALL_PACKAGE appropriately.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-dbusmock_0.25.0.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/python/python3-dbusmock_0.25.0.bb +1 -0
@@ -7,6 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=e6a600fd5e1d9cbde2d983680233ad02" SRC_URI[sha256sum] = "6f4ce7650ecbb022684dc158df720e199635f3a3df75f7020f4fe8f6ff0394db" PYPI_PACKAGE = "python-dbusmock" +PIP_INSTALL_PACKAGE = "python_dbusmock" inherit pypi setuptools3
Signature · 1 line
2.30.2
PATCH v2 30/32 python3-mako: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-mako_1.1.6.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-mako_1.1.6.bb +1 -1
@@ -6,7 +6,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=943eb67718222db21d44a4ef1836675f" PYPI_PACKAGE = "Mako" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta SRC_URI[sha256sum] = "4e9e345a41924a954251b95b4b28e14a301145b544901332e658907a7464b6b2"
Signature · 1 line
2.30.2
PATCH v2 31/32 python3-packaging: inherit setuptools_build_meta
In reply to Tim OrlingUpstream provides a pyproject.toml which declares the setuptools.build_meta backend for PEP-517 packaging.
[YOCTO #14638]
--- meta/recipes-devtools/python/python3-packaging_21.3.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-packaging_21.3.bb +1 -1
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=faadaedca9251a90b205c9167578ce91" SRC_URI[sha256sum] = "dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb" -inherit pypi setuptools3 +inherit pypi setuptools_build_meta BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.30.2
PATCH v2 32/32 python3-nose: drop recipe
In reply to Tim OrlingThe code has not been touched since 2016 and numerous files still have Python2 syntax code in them. This causes do_compile errors when packaging a wheel (PEP-517 packaging).
Nothing in oe-core depends on python3-nose.
[YOCTO #14638]
--- meta/conf/distro/include/maintainers.inc | 1 - meta/recipes-devtools/python/python-nose.inc | 18 ------------------ .../python/python3-nose_1.3.7.bb | 2 -- 3 files changed, 21 deletions(-) delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
meta/conf/distro/include/maintainers.inc +0 -1
@@ -616,7 +616,6 @@ RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr Kravchuk <open.source@oleksandr-k RECIPE_MAINTAINER:pn-python3-markdown = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER:pn-python3-markupsafe = "Richard Purdie <[email protected]>" RECIPE_MAINTAINER:pn-python3-more-itertools = "Tim Orling <[email protected]>" -RECIPE_MAINTAINER:pn-python3-nose = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER:pn-python3-numpy = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER:pn-python3-packaging = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-pathlib2 = "Tim Orling <[email protected]>"
meta/recipes-devtools/python/python-nose.inc deleted +0 -18
@@ -1,18 +0,0 @@ -SUMMARY = "Extends Python unittest to make testing easier" -HOMEPAGE = "http://readthedocs.org/docs/nose/" -DESCRIPTION = "nose extends the test loading and running features of unittest, \ -making it easier to write, find and run tests." -SECTION = "devel/python" -LICENSE = "LGPL-2.1-only" -LIC_FILES_CHKSUM = "file://lgpl.txt;md5=a6f89e2100d9b6cdffcea4f398e37343" - -SRC_URI[md5sum] = "4d3ad0ff07b61373d2cefc89c5d0b20b" -SRC_URI[sha256sum] = "f1bffef9cbc82628f6e7d7b40d7e255aefaa1adb6a1b1d26c69a8b79e6208a98" - -inherit pypi - -RDEPENDS:${PN} = "\ - ${PYTHON_PN}-unittest \ - " - -BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/python/python3-nose_1.3.7.bb deleted +0 -2
@@ -1,2 +0,0 @@ -inherit setuptools3 -require python-nose.inc
Signature · 1 line
2.30.2
Re: [OE-core] [PATCH v2 32/32] python3-nose: drop recipe
there are packages in meta-python which still need it
meta-networking/recipes-connectivity/netplan/netplan_0.103.bb:RDEPENDS:${PN}
= "python3 python3-core python3-pyyaml python3-netifaces python3-nos
e python3-coverage python3-pycodestyle python3-pyflakes util-linux-libuuid libnetplan"
meta-python/recipes-devtools/python/python3-anyjson_0.3.3.bb:
${PYTHON_PN}-nose \
meta-python/recipes-devtools/python/python3-blinker_1.4.bb:
${PYTHON_PN}-nose \
meta-python/recipes-devtools/python/python3-oauthlib_3.2.0.bb:PACKAGECONFIG[test]
= ",,,${PYTHON_PN}-blinker ${PYTHON_PN}-cryptography ${PYTHON_P N}-nose ${PYTHON_PN}-pyjwt"
I wonder if these recipes can either be also removed or made free of nose module.
Quoted 74 lines · Tim Orling
The code has not been touched since 2016 and numerous files still have Python2 syntax code in them. This causes
do_compileerrors when packaging a wheel (PEP-517 packaging).Nothing in oe-core depends on python3-nose.
[YOCTO #14638]
Signed-off-by Tim Orling <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 - meta/recipes-devtools/python/python-nose.inc | 18 ------------------ .../python/python3-nose_1.3.7.bb | 2 -- 3 files changed, 21 deletions(-) delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bbmeta/conf/distro/include/maintainers.inc +0
-1@@ -616,7 +616,6 @@ RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr Kravchuk <open.source@oleksandr-k RECIPE_MAINTAINER:pn-python3-markdown = "Alexander Kanavin <[email protected]>" RECIPE_MAINTAINER:pn-python3-markupsafe = "Richard Purdie <[email protected]>" RECIPE_MAINTAINER:pn-python3-more-itertools = "Tim Orling <[email protected]>" -RECIPE_MAINTAINER:pn-python3-nose = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER:pn-python3-numpy = "Oleksandr Kravchuk <[email protected]>" RECIPE_MAINTAINER:pn-python3-packaging = "Tim Orling <[email protected]>" RECIPE_MAINTAINER:pn-python3-pathlib2 = "Tim Orling <[email protected]>"meta/recipes-devtools/python/python-nose.inc deleted +0
-18@@ -1,18 +0,0 @@ -SUMMARY = "Extends Python unittest to make testing easier" -HOMEPAGE = "http://readthedocs.org/docs/nose/" -DESCRIPTION = "nose extends the test loading and running features of unittest, \ -making it easier to write, find and run tests." -SECTION = "devel/python" -LICENSE = "LGPL-2.1-only" -LIC_FILES_CHKSUM = "file://lgpl.txt;md5=a6f89e2100d9b6cdffcea4f398e37343" - -SRC_URI[md5sum] = "4d3ad0ff07b61373d2cefc89c5d0b20b" -SRC_URI[sha256sum] = "f1bffef9cbc82628f6e7d7b40d7e255aefaa1adb6a1b1d26c69a8b79e6208a98" - -inherit pypi - -RDEPENDS:${PN} = "\ - ${PYTHON_PN}-unittest \ - " - -BBCLASSEXTEND = "native nativesdk"meta/recipes-devtools/python/python3-nose_1.3.7.bb deleted +0
-3@@ -1,2 +0,0 @@ -inherit setuptools3 -require python-nose.inc --2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162108): https://lists.openembedded.org/g/openembedded-core/message/162108
Mute This Topic: https://lists.openembedded.org/mt/89304054/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 32/32] python3-nose: drop recipe
Quoted 16 lines · Khem Raj
there are packages in meta-python which still need it
meta-networking/recipes-connectivity/netplan/netplan_0.103.bb:
RDEPENDS:${PN}
= "python3 python3-core python3-pyyaml python3-netifaces python3-nos e python3-coverage python3-pycodestyle python3-pyflakes util-linux-libuuid libnetplan"
meta-python/recipes-devtools/python/python3-anyjson_0.3.3.bb:
${PYTHON_PN}-nose \
meta-python/recipes-devtools/python/python3-blinker_1.4.bb:
${PYTHON_PN}-nose \
meta-python/recipes-devtools/python/python3-oauthlib_3.2.0.bb:
PACKAGECONFIG[test]
= ",,,${PYTHON_PN}-blinker ${PYTHON_PN}-cryptography ${PYTHON_P N}-nose ${PYTHON_PN}-pyjwt"
Quoted 4 lines
I wonder if these recipes can either be also removed or made free of nose module.
I am thinking the same thing. While we could probably run 2to3 on the nose plugins
that are the legacy python2 code, I would rather do the right thing.
In some cases, pytest can be a drop in replacement for nosetests, but not always.
If anyone has any other ideas, please chime in. I am all ears.
Quoted 91 lines · Tim Orling
Quoted 18 lines
The code has not been touched since 2016 and numerous files still have Python2 syntax code in them. This causes
do_compileerrors when packaging a wheel (PEP-517 packaging).Nothing in oe-core depends on python3-nose.
[YOCTO #14638]
Signed-off-by Tim Orling <[email protected]>--- meta/conf/distro/include/maintainers.inc | 1 - meta/recipes-devtools/python/python-nose.inc | 18 ------------------ .../python/python3-nose_1.3.7.bb | 2 -- 3 files changed, 21 deletions(-) delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bbReview
Quoted 1 line
(quoted diff) +0
-0b/meta/conf/distro/include/maintainers.inc
meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr
Quoted 4 lines
index
7a65612b7c6..333685cd8b8100644meta/conf/distro/include/maintainers.inc +0
-0@@ -616,7 +616,6 @@ RECIPE_MAINTAINER:pn-python3-mako = "OleksandrKravchuk <open.source@oleksandr-k
meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 616
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-markdown = "Alexander Kanavin <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 617
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-markupsafe = "Richard Purdie <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 618
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-more-itertools = "Tim Orling <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 619
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-1-RECIPE_MAINTAINER:pn-python3-nose = "Oleksandr Kravchuk <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 619
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-numpy = "Oleksandr Kravchuk <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 620
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-packaging = "Tim Orling <meta/conf/distro/include/maintainers.inc RECIPE_MAINTAINER:pn-python3-mako = "Oleksandr · line 621
Quoted 1 line
meta/conf/distro/include/maintainers.inc +0
-0RECIPE_MAINTAINER:pn-python3-pathlib2 = "Tim Orling <Review
Quoted 1 line
(quoted diff) +0
-0b/meta/recipes-devtools/python/python-nose.inc
meta/recipes-devtools/python/python-nose.inc line 3
Quoted 8 lines
deleted file mode 100644
index 08c380d637c..00000000000meta/recipes-devtools/python/python-nose.inc deleted +0
-3@@ -1,18 +0,0 @@ -SUMMARY = "Extends Python unittest to make testing easier" -HOMEPAGE = "http://readthedocs.org/docs/nose/" -DESCRIPTION = "nose extends the test loading and running features ofunittest, \
meta/recipes-devtools/python/python-nose.inc
Quoted 4 lines
meta/recipes-devtools/python/python-nose.inc deleted +0
-4-making it easier to write, find and run tests." -SECTION = "devel/python" -LICENSE = "LGPL-2.1-only" -LIC_FILES_CHKSUM ="file://lgpl.txt;md5=a6f89e2100d9b6cdffcea4f398e37343"
meta/recipes-devtools/python/python-nose.inc
Quoted 3 lines
meta/recipes-devtools/python/python-nose.inc deleted +0
-3- -SRC_URI[md5sum] = "4d3ad0ff07b61373d2cefc89c5d0b20b" -SRC_URI[sha256sum] ="
f1bffef9cbc82628f6e7d7b40d7e255aefaa1adb6a1b1d26c69a8b79e6208a98"Review
Quoted 9 lines
meta/recipes-devtools/python/python-nose.inc deleted +0
-8- -inherit pypi - -RDEPENDS:${PN} = "\ - ${PYTHON_PN}-unittest \ - " - -BBCLASSEXTEND = "native nativesdk"(quoted diff) +0
-0b/meta/recipes-devtools/python/python3-nose_1.3.7.bb
Quoted 11 lines
deleted file mode 100644
index 13dbf96179a..00000000000meta/recipes-devtools/python/python3-nose_1.3.7.bb deleted +0
-3@@ -1,2 +0,0 @@ -inherit setuptools3 -require python-nose.inc --2.30.2
Quoted 3 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162108):https://lists.openembedded.org/g/openembedded-core/message/162108
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/89304054/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
this is going into dep loops Try
bitbake python3-wheel-native
Quoted 180 lines · Tim Orling
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS (at a minimum 2 years). This seems to be a compelling enough reason to bring in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allows all other recipes to be installed with pip, as intended by upstream.Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packages which usesetuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than the deprecated 'setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip rather than the deprecated 'setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend should instead use of the new PEP-517 classes. This function is still in progress.
During testing, recipes which use python3-setuptools-scm are failing to determine the version of the python package, For several recipes, the fix was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes are still going to fail for the short term, until the root cause can be determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at this time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than --target) and also passing --prefix so that scripts are (usually) installed in the proper location. pip install also 'helpfully' overwrites python3 intepreter with nativepython3 in #!, so add a fix to return the values to what is actually correct (similar to what is done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheelsTim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++- .../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++ .../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + .../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb create mode 100644 meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} (86%)--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162076): https://lists.openembedded.org/g/openembedded-core/message/162076
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 3 lines · Khem Raj
this is going into dep loops Try
bitbake python3-wheel-native
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () {
# We need to bootstrap python3-wheel-native
install -d ${D}${PYTHON_SITEPACKAGES_DIR}
PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl"
unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \
bbfatal_log "Failed to install"}Quoted 203 lines · Tim Orling
Quoted 17 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS (at a minimum 2 years). This seems to be a compelling enough reason to bring in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allows all other recipes to be installed with pip, as intended by upstream.Quoted 4 lines
Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packages whichuse
Quoted 6 lines
setuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than thedeprecated
Quoted 12 lines
'
setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip rather than the deprecated 'setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Quoted 7 lines
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend should instead use of the new PEP-517 classes. This function is still in
progress.
Quoted 4 lines
During testing, recipes which use python3-setuptools-scm are failing to determine the version of the python package, For several recipes, the fix was to use pyproject.toml and PEP-517 build backends. To be sure, other
recipes
Quoted 10 lines
are still going to fail for the short term, until the root cause can be determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at this time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than
--target)
Quoted 1 line
and also passing --prefix so that scripts are (usually) installed in the
proper
Quoted 1 line
location. pip install also 'helpfully' overwrites python3 intepreter
with nativepython3
Quoted 1 line
in #!, so add a fix to return the values to what is actually correct
(similar to what is
Quoted 1 line
done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Quoted 8 lines
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:Quoted 7 lines
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheelshttp://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Quoted 73 lines
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++- .../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++ .../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 +Quoted 8 lines
.../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
Quoted 2 lines
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patchQuoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patchQuoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patchQuoted 2 lines
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb =>bmap-tools_git.bb} (86%)
Quoted 8 lines
--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162076):https://lists.openembedded.org/g/openembedded-core/message/162076
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 5 lines · Tim Orling
Quoted 1 line · Khem Raj
Quoted 8 lines
Quoted 3 lines
this is going into dep loops Try
bitbake python3-wheel-native
I have been trying whats on master-next
Quoted 194 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 185 lines · Tim Orling
Quoted 180 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS (at a minimum 2 years). This seems to be a compelling enough reason to bring in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allows all other recipes to be installed with pip, as intended by upstream.Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packages which usesetuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than the deprecated 'setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip rather than the deprecated 'setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend should instead use of the new PEP-517 classes. This function is still in progress.
During testing, recipes which use python3-setuptools-scm are failing to determine the version of the python package, For several recipes, the fix was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes are still going to fail for the short term, until the root cause can be determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at this time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than --target) and also passing --prefix so that scripts are (usually) installed in the proper location. pip install also 'helpfully' overwrites python3 intepreter with nativepython3 in #!, so add a fix to return the values to what is actually correct (similar to what is done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheelsTim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++- .../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++ .../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + .../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb create mode 100644 meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} (86%)--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162076): https://lists.openembedded.org/g/openembedded-core/message/162076
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 11 lines · Khem Raj
Quoted 10 lines · Tim Orling
Quoted 4 lines · Khem Raj
this is going into dep loops Try
bitbake python3-wheel-native
Quoted 8 lines
Quoted 2 lines
It shouldn’t
Quoted 1 line
I have been trying whats on master-next
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a hunch.
Quoted 226 lines
Quoted 22 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 13 lines · Tim Orling
Quoted 8 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS
(at
Quoted 1 line
Quoted 1 line
Quoted 1 line
a minimum 2 years). This seems to be a compelling enough reason to
bring
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allowsall
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
other recipes to be installed with pip, as intended by upstream.
Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packageswhich use
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
setuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than thedeprecated
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
'
setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip ratherthan
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
the deprecated '
setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend
should
Quoted 1 line
Quoted 1 line
Quoted 1 line
instead use of the new PEP-517 classes. This function is still in
progress.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
During testing, recipes which use python3-setuptools-scm are failing
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
determine the version of the python package, For several recipes, the
fix
Quoted 1 line
Quoted 1 line
Quoted 1 line
was to use pyproject.toml and PEP-517 build backends. To be sure,
other recipes
Quoted 1 line
Quoted 1 line
Quoted 1 line
are still going to fail for the short term, until the root cause can
be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at
this
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than
--target)
Quoted 1 line
Quoted 1 line
Quoted 1 line
and also passing --prefix so that scripts are (usually) installed in
the proper
Quoted 1 line
Quoted 1 line
Quoted 1 line
location. pip install also 'helpfully' overwrites python3 intepreter
with nativepython3
Quoted 1 line
Quoted 1 line
Quoted 1 line
in #!, so add a fix to return the values to what is actually correct
(similar to what is
Quoted 1 line
Quoted 1 line
Quoted 1 line
done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheelshttp://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++-Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100++++++++++++++++++
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
.../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb =>bmap-tools_git.bb} (86%)
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162076):https://lists.openembedded.org/g/openembedded-core/message/162076
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsubQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 17 lines · Tim Orling
Quoted 13 lines · Khem Raj
Quoted 12 lines · Tim Orling
Quoted 4 lines · Khem Raj
this is going into dep loops Try
bitbake python3-wheel-native
It shouldn’t
Quoted 10 lines
Quoted 6 lines
Quoted 1 line
I have been trying whats on master-next
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a hunch.
Ah. I bet you have meta-python in the bblayers and therefore python3-wheel recipe from meta-python is winning.
Quoted 231 lines
Quoted 230 lines
DEPENDS:remove:class-native = "python3-pip-native"
Quoted 21 lines
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 13 lines · Tim Orling
Quoted 8 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS
(at
Quoted 1 line
Quoted 1 line
Quoted 1 line
a minimum 2 years). This seems to be a compelling enough reason to
bring
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and
python3-pip-native,
Quoted 1 line
Quoted 1 line
Quoted 1 line
simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allowsall
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
other recipes to be installed with pip, as intended by upstream.
Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packageswhich use
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
setuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than thedeprecated
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
'
setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip ratherthan
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the deprecated '
setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because
it
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Missing from this series is documentation of the new features,
classes
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend
should
Quoted 1 line
Quoted 1 line
Quoted 1 line
instead use of the new PEP-517 classes. This function is still in
progress.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
During testing, recipes which use python3-setuptools-scm are failing
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
determine the version of the python package, For several recipes,
the fix
Quoted 1 line
Quoted 1 line
Quoted 1 line
was to use pyproject.toml and PEP-517 build backends. To be sure,
other recipes
Quoted 1 line
Quoted 1 line
Quoted 1 line
are still going to fail for the short term, until the root cause can
be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at
this
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than
--target)
Quoted 1 line
Quoted 1 line
Quoted 1 line
and also passing --prefix so that scripts are (usually) installed in
the proper
Quoted 1 line
Quoted 1 line
Quoted 1 line
location. pip install also 'helpfully' overwrites python3 intepreter
with nativepython3
Quoted 1 line
Quoted 1 line
Quoted 1 line
in #!, so add a fix to return the values to what is actually correct
(similar to what is
Quoted 1 line
Quoted 1 line
Quoted 1 line
done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheelshttp://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++-Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100++++++++++++++++++
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
.../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb =>bmap-tools_git.bb} (86%)
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162076):https://lists.openembedded.org/g/openembedded-core/message/162076
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected] Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 20 lines · Tim Orling
Quoted 16 lines · Khem Raj
Quoted 13 lines · Tim Orling
Quoted 4 lines · Khem Raj
this is going into dep loops Try
bitbake python3-wheel-native
Quoted 6 lines
Quoted 2 lines
I have been trying whats on master-next
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a hunch.
This is because python3-wheel was moved to core and modified but not deleted from meta-python at same time. I have sent a patch for that
Quoted 195 lines
Quoted 195 lines
Quoted 195 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 185 lines · Tim Orling
Quoted 180 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS (at a minimum 2 years). This seems to be a compelling enough reason to bring in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allows all other recipes to be installed with pip, as intended by upstream.Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packages which usesetuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than the deprecated 'setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip rather than the deprecated 'setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend should instead use of the new PEP-517 classes. This function is still in progress.
During testing, recipes which use python3-setuptools-scm are failing to determine the version of the python package, For several recipes, the fix was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes are still going to fail for the short term, until the root cause can be determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at this time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than --target) and also passing --prefix so that scripts are (usually) installed in the proper location. pip install also 'helpfully' overwrites python3 intepreter with nativepython3 in #!, so add a fix to return the values to what is actually correct (similar to what is done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheelsTim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++- .../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++ .../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + .../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb create mode 100644 meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} (86%)--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162076): https://lists.openembedded.org/g/openembedded-core/message/162076
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 25 lines · Khem Raj
Quoted 11 lines · Tim Orling
Quoted 7 lines · Khem Raj
Quoted 5 lines · Tim Orling
Quoted 1 line · Khem Raj
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
this is going into dep loops Try
bitbake python3-wheel-native
It shouldn’t
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
I have been trying whats on master-next
Quoted 8 lines
Quoted 3 lines
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a hunch.
This is because python3-wheel was moved to core and modified but not deleted from meta-python at same time. I have sent a patch for that
Already sent one:
https://patchwork.yoctoproject.org/project/oe/patch/[email protected]/
Quoted 240 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 5 lines
On Mon, Feb 21, 2022 at 1:23 PM Tim Orling <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages with
setup.pyinstall isdeprecated.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
We have a choice with the 'kirkstone' LTS release of introducing
this
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the
LTS (at
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
a minimum 2 years). This seems to be a compelling enough reason to
bring
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and
python3-pip-native,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. Thisallows all
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
other recipes to be installed with pip, as intended by upstream.
Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packageswhich use
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
setuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than thedeprecated
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
'
setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with piprather than
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the deprecated '
setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps)
because it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
proves to be brittle in our environment without providing full
wheel
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than
our
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
normal workflow of requiring dependencies to be defined in a
recipe.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Missing from this series is documentation of the new features,
classes
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend
should
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
instead use of the new PEP-517 classes. This function is still in
progress.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
During testing, recipes which use python3-setuptools-scm are
failing to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
determine the version of the python package, For several recipes,
the fix
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
was to use pyproject.toml and PEP-517 build backends. To be sure,
other recipes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
are still going to fail for the short term, until the root cause
can be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core
at this
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather
than --target)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
and also passing --prefix so that scripts are (usually) installed
in the proper
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
location. pip install also 'helpfully' overwrites python3
intepreter with nativepython3
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in #!, so add a fix to return the values to what is actually
correct (similar to what is
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++-Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100++++++++++++++++++
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
.../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 +Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb =>bmap-tools_git.bb} (86%)
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162076):https://lists.openembedded.org/g/openembedded-core/message/162076
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Mute This Topic:
https://lists.openembedded.org/mt/89304007/1997914
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Group Owner: [email protected] Unsubscribe:
https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
Right now approx 150 out of 470 recipes [1] in meta-python are directly failing, perhaps there will be more once these are fixed then there are other layers which might be carrying some impacted python modules. Also YP compatibility is broken due to python3-nose removal. If we are planning
to include this in the upcoming release,we would need some commitment and strategy to fix these and other ripple effects throughout the ecosystem especially since this is an LTS release. I wish this changeset was proposed in a release prior to LTS release, so it could have given enough time to adjust for the ecosystem. This is will take some effort in stabilizing
I also wonder, what do we lose if we do not put this into the LTS release?
[1] https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/1460
Quoted 235 lines · Tim Orling
Quoted 25 lines · Khem Raj
Quoted 23 lines · Tim Orling
Quoted 18 lines · Khem Raj
Quoted 5 lines · Tim Orling
Quoted 1 line · Khem Raj
Quoted 8 lines
Quoted 3 lines
this is going into dep loops Try
bitbake python3-wheel-native
I have been trying whats on master-next
Quoted 7 lines
Quoted 3 lines
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a hunch.
This is because python3-wheel was moved to core and modified but not deleted from meta-python at same time. I have sent a patch for that
Already sent one:
https://patchwork.yoctoproject.org/project/oe/patch/[email protected]/Quoted 196 lines
Quoted 196 lines
Quoted 196 lines
Quoted 196 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 185 lines · Tim Orling
Quoted 180 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyand wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages withsetup.pyinstall is deprecated.We have a choice with the 'kirkstone' LTS release of introducing this series (and fixing any fall out during the stabilization period) or living with deprecated Python packaging for the duration of the LTS (at a minimum 2 years). This seems to be a compelling enough reason to bring in this series, even if it results in a slight slip of the 'feature freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and python3-pip-native, simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. This allows all other recipes to be installed with pip, as intended by upstream.Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliant packages which usesetuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built by other classes.The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather than the deprecated 'setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with pip rather than the deprecated 'setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps) because it proves to be brittle in our environment without providing full wheel cache that seems like overhead that we do not need. We also disable fetching wheels from pypi (--no-index). This is no different than our normal workflow of requiring dependencies to be defined in a recipe.
Missing from this series is documentation of the new features, classes and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits setuptools3 but has a pyproject.toml and defines a build-system.build-backend should instead use of the new PEP-517 classes. This function is still in progress.
During testing, recipes which use python3-setuptools-scm are failing to determine the version of the python package, For several recipes, the fix was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes are still going to fail for the short term, until the root cause can be determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into oe-core at this time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather than --target) and also passing --prefix so that scripts are (usually) installed in the proper location. pip install also 'helpfully' overwrites python3 intepreter with nativepython3 in #!, so add a fix to return the values to what is actually correct (similar to what is done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheelsTim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++- .../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++ .../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + .../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644 meta/recipes-devtools/python/python-subunit.inc create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb create mode 100644 meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patch create mode 100644 meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patch create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb => bmap-tools_git.bb} (86%)--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162076): https://lists.openembedded.org/g/openembedded-core/message/162076
Mute This Topic: https://lists.openembedded.org/mt/89304007/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 4 lines · Khem Raj
Right now approx 150 out of 470 recipes [1] in meta-python are directly failing,
perhaps there will be more once these are fixed then there are other layers which might be carrying some impacted python modules.
The issue has mostly been detection of the wheel filename and this should be improved by:
https://lore.kernel.org/openembedded-core/[email protected]/T/#t
Also YP compatibility
Quoted 1 line
is broken due to python3-nose removal.
We can try to run 2to3 on the python3-nose plugins or remove them surgically to reintroduce cleanly Python 3 only python3-nose. I propose that we move it to meta-python even so because nothing in oe-core depends on nose and the
code has been completely bit-rotting since 2016. It could not be built anymore in oe-core and something had to give.
Quoted 7 lines
If we are planning
to include this in the upcoming release,we would need some commitment and strategy to fix these and other ripple effects throughout the ecosystem especially since this is an LTS release. I wish this changeset was proposed in a release prior to LTS release, so it could have given enough time to adjust for the ecosystem. This is will take some effort in stabilizing
This has been months of work on my part to attempt to get us to where upstream
Python Packaging Authority has been pushing since PEP-517 was announced in 2015.
We are very far behind the curve of upstream Python packaging.
While I too would love to have had this ready sooner, that was not possible for many reasons.
But no matter what, this would have been painful until recipes could be migrated, even if had
been available at M1.
Quoted 2 lines
I also wonder, what do we lose if we do not put this into the LTS release?
We already have recipes in oe-core such as python3-jsonschema which can no longer be upgraded
without moving to pyproject.toml based package builds. I'm sure there are others in meta-python and
other layers in a similar story. We once again are in a situation that is a bad image for the project because
we are using the now deprecated "setup.py install" and still building eggs. To step into an LTS release
with the python packaging already entirely deprecated is just not a good impression.
I understand your frustration with the build failures, but it has been a very solid week of getting oe-core
stabilized (see
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/3280).
I am running a build now for meta-python with the improved wheel filename patch as well as a patch to
check for whether the current recipe already has pyproject.toml back-end declared:
https://lore.kernel.org/openembedded-core/[email protected]/T/#u
Quoted 1 line
[1] https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/1460
Quoted 293 lines · Tim Orling
Quoted 12 lines · Khem Raj
Quoted 8 lines · Tim Orling
Quoted 6 lines · Khem Raj
Quoted 2 lines
On Tue, Feb 22, 2022 at 10:21 AM Tim Orling <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
On Tue, Feb 22, 2022 at 9:22 AM Khem Raj <[email protected]>
Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
this is going into dep loops Try
bitbake python3-wheel-native
It shouldn’t
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
I have been trying whats on master-next
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a
hunch.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
This is because python3-wheel was moved to core and modified but not deleted from meta-python at same time. I have sent a patch for that
Already sent one:
https://patchwork.yoctoproject.org/project/oe/patch/[email protected]/
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 5 lines
On Mon, Feb 21, 2022 at 1:23 PM Tim Orling <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyandQuoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
wheels rather than eggs. The main place this is documented is in PEP-517[1]. Installing packages with
setup.pyinstall isdeprecated.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
We have a choice with the 'kirkstone' LTS release of
introducing this
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
series (and fixing any fall out during the stabilization
period) or
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
living with deprecated Python packaging for the duration of the
LTS (at
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
a minimum 2 years). This seems to be a compelling enough reason
to bring
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in this series, even if it results in a slight slip of the
'feature
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and
python3-pip-native,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. Thisallows all
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
other recipes to be installed with pip, as intended by upstream.
Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliantpackages which use
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
setuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built byother
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
classes.
The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather thanthe deprecated
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
'
setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with piprather than
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the deprecated '
setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps)
because it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
proves to be brittle in our environment without providing full
wheel
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
cache that seems like overhead that we do not need. We also
disable
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
fetching wheels from pypi (--no-index). This is no different
than our
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
normal workflow of requiring dependencies to be defined in a
recipe.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Missing from this series is documentation of the new features,
classes
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a helper function to warn that a recipe which currently inherits
setuptools3
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
but has a pyproject.toml and defines a
build-system.build-backend should
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
instead use of the new PEP-517 classes. This function is still
in progress.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
During testing, recipes which use python3-setuptools-scm are
failing to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
determine the version of the python package, For several
recipes, the fix
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
was to use pyproject.toml and PEP-517 build backends. To be
sure, other recipes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
are still going to fail for the short term, until the root
cause can be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into
oe-core at this
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather
than --target)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
and also passing --prefix so that scripts are (usually)
installed in the proper
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
location. pip install also 'helpfully' overwrites python3
intepreter with nativepython3
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in #!, so add a fix to return the values to what is actually
correct (similar to what is
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
done in setuptools3.bbclass). Also several recipes still needed
PIP_INSTALL_PACKAGEdefined.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++-Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100++++++++++++++++++
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
.../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + .../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644 meta/recipes-devtools/python/python-nose.inc delete mode 100644meta/recipes-devtools/python/python-subunit.inc
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb =>bmap-tools_git.bb} (86%)
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162076):https://lists.openembedded.org/g/openembedded-core/message/162076
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Mute This Topic:
https://lists.openembedded.org/mt/89304007/1997914
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Group Owner: [email protected] Unsubscribe:
https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 20 lines · Tim Orling via lists.openembedded.org
Quoted 5 lines · Khem Raj
Right now approx 150 out of 470 recipes [1] in meta-python are directly failing,
perhaps there will be more once these are fixed then there are other layers
which might be carrying some impacted python modules.The issue has mostly been detection of the wheel filename and this should be improved by:
https://lore.kernel.org/openembedded-core/[email protected]/T/#t
Also YP compatibility
Quoted 1 line
is broken due to python3-nose removal.
Quoted 7 lines
We can try to run 2to3 on the python3-nose plugins or remove them surgically to reintroduce cleanly Python 3 only python3-nose. I propose that we move it to meta-python even so because nothing in oe-core depends on nose and the
code has been completely bit-rotting since 2016. It could not be built anymore in oe-core and something had to give.
Perhaps we can switch to this repo:
https://github.com/atsb/nose-py3
Quoted 359 lines
Quoted 11 lines
If we are planning
to include this in the upcoming release,we would need some commitment and strategy to fix these and other ripple effects throughout the ecosystem
especially since this is an LTS release. I wish this changeset was proposed
in a release prior to LTS release, so it could have given enough time to adjust for the ecosystem. This is will take some effort in stabilizingThis has been months of work on my part to attempt to get us to where upstream
Python Packaging Authority has been pushing since PEP-517 was announced in 2015.
We are very far behind the curve of upstream Python packaging.While I too would love to have had this ready sooner, that was not possible for many reasons.
But no matter what, this would have been painful until recipes could be migrated, even if had
been available at M1.Quoted 2 lines
I also wonder, what do we lose if we do not put this into the LTS release?
We already have recipes in oe-core such as python3-jsonschema which can no longer be upgraded
without moving to pyproject.toml based package builds. I'm sure there are others in meta-python and
other layers in a similar story. We once again are in a situation that is a bad image for the project because
we are using the now deprecated "setup.pyinstall" and still building eggs. To step into an LTS release
with the python packaging already entirely deprecated is just not a good impression.I understand your frustration with the build failures, but it has been a very solid week of getting oe-core
stabilized (see
https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/3280).I am running a build now for meta-python with the improved wheel filename patch as well as a patch to
check for whether the current recipe already has pyproject.toml back-end declared:https://lore.kernel.org/openembedded-core/[email protected]/T/#u
Quoted 1 line
[1]
Quoted 276 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/1460
Quoted 6 lines · Tim Orling
Quoted 2 lines · Khem Raj
On Tue, Feb 22, 2022 at 10:40 AM Tim Orling <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 4 lines
On Tue, Feb 22, 2022 at 10:24 AM Khem Raj <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Tue, Feb 22, 2022 at 10:21 AM Tim Orling <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
On Tue, Feb 22, 2022 at 9:22 AM Khem Raj <[email protected]>
Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
this is going into dep loops Try
bitbake python3-wheel-native
It shouldn’t
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
I have been trying whats on master-next
I can't reproduce. Can you point me at an error log please? It might have something to do with sstate cache, but that's just a
hunch.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
This is because python3-wheel was moved to core and modified but not deleted from meta-python at same time. I have sent a patch for that
Already sent one:
https://patchwork.yoctoproject.org/project/oe/patch/[email protected]/
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
DEPENDS:remove:class-native = "python3-pip-native"
do_install:class-native () { # We need to bootstrap python3-wheel-native install -d ${D}${PYTHON_SITEPACKAGES_DIR} PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl" unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \ bbfatal_log "Failed to install" }Quoted 5 lines
On Mon, Feb 21, 2022 at 1:23 PM Tim Orling <[email protected]>
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Upstream Python is moving to pyproject.toml rather than
setup.pyandQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
wheels rather than eggs. The main place this is documented is
in
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
PEP-517[1]. Installing packages with
setup.pyinstall isdeprecated.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
We have a choice with the 'kirkstone' LTS release of
introducing this
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
series (and fixing any fall out during the stabilization
period) or
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
living with deprecated Python packaging for the duration of
the LTS (at
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
a minimum 2 years). This seems to be a compelling enough
reason to bring
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in this series, even if it results in a slight slip of the
'feature
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
freeze', which is officially tomorrow, Monday 21 Feb, 2022.
This series bootstraps python3-setuptools-native, python3-flit-core-native, python3-wheel-native and
python3-pip-native,
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
simply unzipping the wheel into
PYTHON_SITEPACKAGES_DIR. Thisallows all
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
other recipes to be installed with pip, as intended by
upstream.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Three new classes are introduced:
(1)flit_corewhich builds PEP-517 compliant packages which use flit_core.buildapi as their build-backend.
(2)setuptools_build_metawhich builds PEP-517 compliantpackages which use
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
setuptools_build_metaas their build-backend.
(3)pip_install_wheel, which simply installs wheels built byother
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
classes.
The original setuptools3.bbclass
do_compileis modified to 'setup.pybdist_wheel' (from python3-wheel-native) rather thanthe deprecated
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
'
setup.pybuild' mechanism.The original setuptools3.bbclass
do_installis modified to use the pip_install_wheel.bbclass to install wheels with piprather than
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
the deprecated '
setup.pyinstall' mechanism.For pip install, we disable dependency checking (--no-deps)
because it
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
proves to be brittle in our environment without providing full
wheel
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
cache that seems like overhead that we do not need. We also
disable
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
fetching wheels from pypi (--no-index). This is no different
than our
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
normal workflow of requiring dependencies to be defined in a
recipe.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Missing from this series is documentation of the new features,
classes
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
and variables (this will be done under separate cover).
Also desired and previously discussed in YP meetings is a
helper
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
function to warn that a recipe which currently inherits
setuptools3
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
but has a pyproject.toml and defines a
build-system.build-backend should
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
instead use of the new PEP-517 classes. This function is still
in progress.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
During testing, recipes which use python3-setuptools-scm are
failing to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
determine the version of the python package, For several
recipes, the fix
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
was to use pyproject.toml and PEP-517 build backends. To be
sure, other recipes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
are still going to fail for the short term, until the root
cause can be
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
determined. This was not seen again in v2 builds.
Another PEP-517 build-backend, poetry.core.masonry.api has been identified in recipes in meta-python, but not in oe-core. The python3-poetry-core recipe is WIP, but brings in a number of dependencies that are perhaps not worth it to bring into
oe-core at this
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
time. Rather we can support these recipes in meta-python?
Fixes in v2 include passing --root=${D} to pip install (rather
than --target)
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
and also passing --prefix so that scripts are (usually)
installed in the proper
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
location. pip install also 'helpfully' overwrites python3
intepreter with nativepython3
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in #!, so add a fix to return the values to what is actually
correct (similar to what is
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
done in setuptools3.bbclass). Also several recipes still
needed
PIP_INSTALL_PACKAGEdefined.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Also added self as maintainer of new recipes.
[YOCTO #14638]
[1] https://www.python.org/dev/peps/pep-0517/
The following changes since commit
41ae26ecab731f2fb6593df143476c94183be209:Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
releases: update to include 3.4.2 (2022-02-21 16:46:22 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/git/poky-contribtimo/bootstrap-wheels
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Quoted 61 lines
Tim Orling (32):
python3-wheel: move 0.37.0 from meta-python
python3-flit-core: add recipe for 3.6.0 flit_core.bbclass: add helper for newer python packaging
python3-wheel: inheritflit_corepip_install_wheel.bbclass: add helper class setuptools_build_meta.bbclass: add helper class
python3-pip: inheritsetuptools_build_meta
python3-attrs: inheritsetuptools_build_meta
python3-git: inheritsetuptools_build_meta
python3-pytest: inheritsetuptools_build_meta
python3-setuptools-scm: inheritsetuptools_build_meta
python3-zipp: inheritsetuptools_build_meta
python3-iniconfig: inheritsetuptools_build_meta
python3-py: inheritsetuptools_build_meta
python3-pluggy: inheritsetuptools_build_meta
python3-setuptools: inheritsetuptools_base_metasetuptools3.bbclass: refactor for wheels
python3-more-itertools: setPIP_INSTALL_PACKAGE
meson: inheritsetuptools_build_meta
python3-libarchive-d: setPIP_INSTALL_PACKAGE
python3-smartypants: patch hash bang to python3
python3-scons: merge -native recipe
python3-subunit: merge inc; setPIP_INSTALL_PACKAGE
python3-magic: setPIP_INSTALL_PACKAGE
bmap-tools: setPIP_INSTALL_PACKAGE, BASEVER
asciidoc: setPIP_INSTALL_PACKAGE
gi-docgen: setPIP_INSTALL_PACKAGE
numpy: setPIP_INSTALL_PACKAGE
python3-dbusmock: setPIP_INSTALL_PACKAGE
python3-mako: inheritsetuptools_build_meta
python3-packaging: inheritsetuptools_build_meta
python3-nose: drop recipemeta/classes/flit_core.bbclass | 16 +++ meta/classes/pip_install_wheel.bbclass | 37 +++++++ meta/classes/setuptools3.bbclass | 20 ++-- meta/classes/setuptools_build_meta.bbclass | 18 ++++ meta/conf/distro/include/maintainers.inc | 3 +- meta/recipes-devtools/meson/meson_0.61.2.bb | 4 +- meta/recipes-devtools/python/python-nose.inc | 18 ---- .../python/python-subunit.inc | 14 --- .../python/python3-attrs_21.4.0.bb | 2 +- .../python/python3-dbusmock_0.25.0.bb | 1 + .../python/python3-flit-core_3.6.0.bb | 41 +++++++ .../python/python3-git_3.1.26.bb | 2 +- .../python/python3-iniconfig_1.1.1.bb | 4 +- .../python/python3-libarchive-c_4.0.bb | 2 + .../python/python3-magic_0.4.25.bb | 1 + .../python/python3-mako_1.1.6.bb | 2 +- .../python/python3-more-itertools_8.12.0.bb | 2 + .../python/python3-nose_1.3.7.bb | 2 - .../python/python3-numpy_1.22.2.bb | 1 + .../python/python3-packaging_21.3.bb | 2 +- .../python/python3-pip_22.0.3.bb | 19 +++- .../python/python3-pluggy_1.0.0.bb | 2 +- .../python/python3-py_1.11.0.bb | 2 +- .../python/python3-pytest_7.0.1.bb | 4 +- .../python/python3-scons-native_4.3.0.bb | 7 -- .../python/python3-scons_4.3.0.bb | 13 ++-Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
.../python/python3-setuptools-scm_6.4.2.bb | 2 +- .../python/python3-setuptools_59.5.0.bb | 19 +++- .../0001-Change-hash-bang-to-python3.patch | 47 ++++++++ .../python/python3-smartypants_2.0.0.bb | 1 + ...001-setup.py-use-vendored-_distutils.patch | 27 +++++ .../python/python3-strict-rfc3339_0.7.bb | 4 +- .../python/python3-subunit_1.4.0.bb | 17 ++- ...roject.toml-from-flit-backend-branch.patch | 100++++++++++++++++++
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
.../python/python3-wheel_0.37.1.bb | 24 +++++ .../python/python3-zipp_3.7.0.bb | 2 +- .../asciidoc/asciidoc_10.1.1.bb | 1 + meta/recipes-gnome/gi-docgen/gi-docgen_git.bb | 1 + .../{bmap-tools_3.6.bb => bmap-tools_git.bb} | 6 +- 39 files changed, 412 insertions(+), 78 deletions(-) create mode 100644 meta/classes/flit_core.bbclass create mode 100644 meta/classes/pip_install_wheel.bbclass create mode 100644 meta/classes/setuptools_build_meta.bbclass delete mode 100644meta/recipes-devtools/python/python-nose.inc
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644meta/recipes-devtools/python/python-subunit.inc
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-nose_1.3.7.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
delete mode 100644 meta/recipes-devtools/python/python3-scons-native_4.3.0.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-smartypants/0001-Change-hash-bang-to-python3.patchQuoted 16 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-strict-rfc3339/0001-setup.py-use-vendored-_distutils.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644
meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patchQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rename meta/recipes-support/bmap-tools/{bmap-tools_3.6.bb =>bmap-tools_git.bb} (86%)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
--
2.30.2-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#162292):
https://lists.openembedded.org/g/openembedded-core/message/162292
Mute This Topic: https://lists.openembedded.org/mt/89304007/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-