Skip to contentSkip to conversation

setuptools3.bbclass: add check for pyproject.toml

PATCH Superseded Tim Orling 3 participants 5 messages Feb 25, 2022 – Feb 26, 2022

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Tim Orling

author
#0 lore raw

PATCH v2 1/2 setuptools3.bbclass: add check for pyproject.toml

From: Tim Orling <[email protected]>

With help from Peter Kjellerstedt <[email protected]> via IRC.

Add a check for pyproject.toml in ${S} and if so check if it has a [build-system] build-backend. Give the user a helpful warning that the recipe should be changed to one of the PEP-517 classes (instead of setuptools3.bbclass).

Add SETUPTOOLS_SKIP_BUILD_BACKEND_CHECK variable to skip this check (and avoid the warning). This is needed for e.g. python3-setuptools-rust-native which does not build cleanly with setuptools_build_meta.bbclass

[YOCTO #14736]

---
Changes in v2:
fix whitespace errors

 meta/classes/setuptools3.bbclass | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
meta/classes/setuptools3.bbclass +32 -0
index 12561340b07..12d662fb49c 100644
@@ -18,6 +18,38 @@ setuptools3_do_configure() {
     :
 }
 
+SETUPTOOLS_SKIP_BUILD_BACKEND_CHECK ?= "0"
+
+python check_for_pyprojecttoml_build_backend() {
+    import os
+    import tomli
+    from pathlib import Path
+
+    if d.getVar('SETUPTOOLS_SKIP_BUILD_BACKEND_CHECK') == "1":
+        bb.debug(3, "Skipping check for build-backend in pyproject.toml")
+        return 0
+    pyprojecttoml_file = Path(d.getVar('S'), 'pyproject.toml')
+    if pyprojecttoml_file.exists():
+        bb.debug(3, "pyproject.toml found: %s" % pyprojecttoml_file)
+        with open(pyprojecttoml_file, "rb") as f:
+            pyprojecttoml_dict = tomli.load(f)
+            build_system = pyprojecttoml_dict["build-system"]
+            if build_system:
+                bb.debug(3, "[build-system] found in pyproject.toml")
+                backend = build_system.get('build-backend')
+                if backend:
+                    bb.debug(3, "build-backend found: %s" % backend)
+                if backend == "flit_core.buildapi":
+                    bb.warn("The source has a pyproject.toml which declares 'flit_core.buildapi' as a build backend, please consider 'inherit flit_core' instead of inheriting setuptools3.")
+                elif backend == "setuptools.build_meta":
+                    bb.warn("The source has a pyproject.toml which declares 'setuptools.build_meta' as a build backend, please consider 'inherit setuptools_build_meta' instead of inheriting setuptools3.")
+                elif backend == "poetry.core.masonry.api":
+                    bb.warn("The source has a pyproject.toml which declares 'poetry.core.masonry.api' as a build backend, please consider 'inherit poetry_core' from meta-python instead of inheriting setuptools3.")
+                else:
+                    bb.warn("The source has a pyproject.toml which declares '%s' as a build backend, but this is not currently supported in oe-core." % backend)
+}
+do_configure[prefuncs] += "check_for_pyprojecttoml_build_backend"
+
 setuptools3_do_compile() {
         cd ${SETUPTOOLS_SETUP_PATH}
         NO_FETCH_BUILD=1 \
Signature · 1 line
2.30.2
author
#1 lore raw

PATCH v2 2/2 pip_install_wheel: improved wheel filename guess

From: Tim Orling <[email protected]>

Rather than only use PYPI_PACKAGE as a guess, fall back on PN for cases where a recipe does not inherit pypi.

Wheels can only have alphanumeric characters in the 'distribution' name [1]. Any other characters are replaced with an underscore. Provide a function to replace dash with underscore.

[1] https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode

---
Changes in v2:
address review comments

 meta/classes/pip_install_wheel.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/pip_install_wheel.bbclass +5 -1
index 8a848c0ebab..9f9feda6ee0 100644
@@ -1,6 +1,10 @@
 DEPENDS:append = " python3-pip-native"
 
-PIP_INSTALL_PACKAGE ?= "${PYPI_PACKAGE}"
+def guess_pip_install_package_name(d):
+    '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
+    return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_')
+
+PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
 PIP_INSTALL_DIST_PATH ?= "${B}/dist"
 PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
 
Signature · 1 line
2.30.2
#2 lore raw

Re: [OE-core] [PATCH v2 2/2] pip_install_wheel: improved wheel filename guess

Quoted 24 lines · Tim Orling

From: Tim Orling <[email protected]>

Rather than only use PYPI_PACKAGE as a guess, fall back on PN for cases where a recipe does not inherit pypi.

Wheels can only have alphanumeric characters in the 'distribution' name [1]. Any other characters are replaced with an underscore. Provide a function to replace dash with underscore.

[1] https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode

---
Changes in v2:
address review comments

  meta/classes/pip_install_wheel.bbclass | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/pip_install_wheel.bbclass +0 -0
index 8a848c0ebab..9f9feda6ee0 100644

meta/classes/pip_install_wheel.bbclass line 5

Quoted 6 lines
meta/classes/pip_install_wheel.bbclass +3 -1
@@ -1,6 +1,10 @@
  DEPENDS:append = " python3-pip-native"
 
-PIP_INSTALL_PACKAGE ?= "${PYPI_PACKAGE}"
+def guess_pip_install_package_name(d):
+    '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
+    return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_')

In my opinion this needs to be BPN not PN - as if you will build python3-foo-native (for native only recipes) it will result in foo_native-1.2.3-*.whl, that just doesn't exit. What disappoints me the most is that none of this comes with unit tests - it broke for a bunch of stuff I maintain this morning, making my morning coffee taste bitter :-(

Quoted 16 lines
meta/classes/pip_install_wheel.bbclass +2 -1
+
+PIP_INSTALL_PACKAGE ?= "${@guess_pip_install_package_name(d)}"
  PIP_INSTALL_DIST_PATH ?= "${B}/dist"
  PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
 
 
 
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.
View/Reply Online (#162335): https://lists.openembedded.org/g/openembedded-core/message/162335
Mute This Topic: https://lists.openembedded.org/mt/89382270/3647476
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Re: [OE-core] [PATCH v2 2/2] pip_install_wheel: improved wheel filename guess

Quoted 28 lines · Konrad Weihmann
Quoted 26 lines · Tim Orling

From: Tim Orling <[email protected]>

Rather than only use PYPI_PACKAGE as a guess, fall back on PN for cases where a recipe does not inherit pypi.

Wheels can only have alphanumeric characters in the 'distribution' name [1]. Any other characters are replaced with an underscore. Provide a function to replace dash with underscore.

[1] https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode

---
Changes in v2:
address review comments

  meta/classes/pip_install_wheel.bbclass | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/pip_install_wheel.bbclass +0 -0
index 8a848c0ebab..9f9feda6ee0 100644
@@ -1,6 +1,10 @@
  DEPENDS:append = " python3-pip-native"
 
Quoted 8 lines

meta/classes/pip_install_wheel.bbclass line 5

Quoted 4 lines
meta/classes/pip_install_wheel.bbclass +3 -1
-PIP_INSTALL_PACKAGE ?= "${PYPI_PACKAGE}"
+def guess_pip_install_package_name(d):
+    '''https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode'''
+    return (d.getVar('PYPI_PACKAGE') or d.getVar('PN')).replace('-', '_')

In my opinion this needs to be BPN not PN - as if you will build python3-foo-native (for native only recipes) it will result in foo_native-1.2.3-*.whl, that just doesn't exit.

Agreed, fix sent.

Quoted 3 lines

What disappoints me the most is that none of this comes with unit tests
- it broke for a bunch of stuff I maintain this morning, making my morning coffee taste bitter :-(

I'd love more tests!

Cheers,

Richard

author
#4 lore raw
In reply to Tim Orling

Hold off on this one

Quoted 40 lines · Tim Orling

From: Tim Orling <[email protected]>

With help from Peter Kjellerstedt <[email protected]> via IRC.

Add a check for pyproject.toml in ${S} and if so check if it has a [build-system] build-backend. Give the user a helpful warning that the recipe should be changed to one of the PEP-517 classes (instead of setuptools3.bbclass).

Add SETUPTOOLS_SKIP_BUILD_BACKEND_CHECK variable to skip this check (and avoid the warning). This is needed for e.g. python3-setuptools-rust-native which does not build cleanly with setuptools_build_meta.bbclass

[YOCTO #14736]

---
Changes in v2:
fix whitespace errors

 meta/classes/setuptools3.bbclass | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
meta/classes/setuptools3.bbclass +7 -0
index 12561340b07..12d662fb49c 100644
@@ -18,6 +18,38 @@ setuptools3_do_configure() {
     :
 }
 
+SETUPTOOLS_SKIP_BUILD_BACKEND_CHECK ?= "0"
+
+python check_for_pyprojecttoml_build_backend() {
+    import os
+    import tomli
+    from pathlib import Path
+

meta/classes/setuptools3.bbclass setuptools3_do_configure() { · line 37

Quoted 10 lines
meta/classes/setuptools3.bbclass +9 -0
+    if d.getVar('SETUPTOOLS_SKIP_BUILD_BACKEND_CHECK') == "1":
+        bb.debug(3, "Skipping check for build-backend in pyproject.toml")
+        return 0
+    pyprojecttoml_file = Path(d.getVar('S'), 'pyproject.toml')
+    if pyprojecttoml_file.exists():
+        bb.debug(3, "pyproject.toml found: %s" % pyprojecttoml_file)
+        with open(pyprojecttoml_file, "rb") as f:
+            pyprojecttoml_dict = tomli.load(f)
+            build_system = pyprojecttoml_dict["build-system"]
 

This needs results in KeyError when the source has pyproject.toml but not [buiid-system],

Quoted 31 lines
meta/classes/setuptools3.bbclass +7 -0
+            if build_system:
+                bb.debug(3, "[build-system] found in pyproject.toml")
+                backend = build_system.get('build-backend')
+                if backend:
+                    bb.debug(3, "build-backend found: %s" % backend)
+                if backend == "flit_core.buildapi":
+                    bb.warn("The source has a pyproject.toml which

declares 'flit_core.buildapi' as a build backend, please consider 'inherit flit_core' instead of inheriting setuptools3.")

meta/classes/setuptools3.bbclass +2 -0
+                elif backend == "setuptools.build_meta":
+                    bb.warn("The source has a pyproject.toml which

declares 'setuptools.build_meta' as a build backend, please consider 'inherit setuptools_build_meta' instead of inheriting setuptools3.")

meta/classes/setuptools3.bbclass +2 -0
+                elif backend == "poetry.core.masonry.api":
+                    bb.warn("The source has a pyproject.toml which

declares 'poetry.core.masonry.api' as a build backend, please consider 'inherit poetry_core' from meta-python instead of inheriting setuptools3.")

meta/classes/setuptools3.bbclass +2 -0
+                else:
+                    bb.warn("The source has a pyproject.toml which

declares '%s' as a build backend, but this is not currently supported in oe-core." % backend)

meta/classes/setuptools3.bbclass +3 -1
+}
+do_configure[prefuncs] += "check_for_pyprojecttoml_build_backend"
+
 setuptools3_do_compile() {
         cd ${SETUPTOOLS_SETUP_PATH}
         NO_FETCH_BUILD=1 \
--

2.30.2