Skip to contentSkip to conversation

Add new python modules for building others

RFC New Zoltán Böszörményi 2 participants 11 messages Mar 10, 2023 1 message not in the archive

lore
Share & tools
lore
#2 lore raw

PATCH 1/4 cmake, cmake-native: Modify download filename

So it won't clash with python3-cmake's download filename from pypi.org which obviously has a diffferent checksum.

---
 meta/recipes-devtools/cmake/cmake.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/cmake/cmake.inc +1 -1
index b7e674c4c4..9005255a23 100644
@@ -16,7 +16,7 @@ LIC_FILES_CHKSUM = "file://Copyright.txt;md5=09069e0fffe4e5eaf6dde04c3b1932e5 \
 
 CMAKE_MAJOR_VERSION = "${@'.'.join(d.getVar('PV').split('.')[0:2])}"
 
-SRC_URI = "https://cmake.org/files/v${CMAKE_MAJOR_VERSION}/cmake-${PV}.tar.gz \
+SRC_URI = "https://cmake.org/files/v${CMAKE_MAJOR_VERSION}/cmake-${PV}.tar.gz;downloadfilename=cmake-src-${PV}.tar.gz \
 "
 
 SRC_URI[sha256sum] = "c026f22cb931dd532f648f087d587f07a1843c6e66a3dfca4fb0ea21944ed33c"
Signature · 1 line
2.39.2
#3 lore raw

PATCH 2/4 python3-scikit-build: New recipe

In reply to Zoltán Böszörményi

Used by python3-cmake and python3-ninja.

---
 .../python/python3-scikit-build_0.16.7.bb     | 21 +++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-scikit-build_0.16.7.bb
meta/recipes-devtools/python/python3-scikit-build_0.16.7.bb new file +21 -0
new file mode 100644
index 0000000000..c033c56bad
@@ -0,0 +1,21 @@
+SUMMARY = "Improved build system generator for Python C/C++/Fortran/Cython extensions"
+LICENSE = "MIT & BSD-2-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=7c96d2b08b3cec6d3c67fb864d1fd8cc"
+
+DEPENDS = "python3-setuptools-scm-native"
+
+PYPI_PACKAGE = "scikit-build"
+
+inherit pypi python_setuptools_build_meta
+SRC_URI[sha256sum] = "a9b9cc7479b71e6c8d434596dfade025253aae23adb22a9a2d85850fd51cecfd"
+
+RDEPENDS:${PN} = " \
+	python3-distro \
+	python3-packaging \
+	python3-setuptools \
+	python3-typing-extensions \
+	python3-wheel \
+	cmake \
+"
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#4 lore raw

PATCH 3/4 python3-cmake: New recipe

In reply to Zoltán Böszörményi

Some python modules rely on "import cmake".

Only build and ship the python parts, and don't download and build cmake from sources. Use the already built cmake instead.

The CMakeLists.txt file is a crippled copy from this cmake python module's sources, removing almost everything, and adding a dummy install target, so do_install() doesn't fail.

The python code is patched so cmake is used from PATH.

---
 .../python/python3-cmake/CMakeLists.txt       | 11 +++++++
 .../python3-cmake/run-cmake-from-path.patch   | 11 +++++++
 .../python/python3-cmake_3.25.2.bb            | 33 +++++++++++++++++++
 3 files changed, 55 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-cmake/CMakeLists.txt
 create mode 100644 meta/recipes-devtools/python/python3-cmake/run-cmake-from-path.patch
 create mode 100644 meta/recipes-devtools/python/python3-cmake_3.25.2.bb
meta/recipes-devtools/python/python3-cmake/CMakeLists.txt new file +11 -0
new file mode 100644
index 0000000000..8ba81970ff
@@ -0,0 +1,11 @@
+cmake_minimum_required(VERSION 3.6)
+
+#
+# For more details, see docs/building.rst
+#
+
+project(CMakePythonDistributions NONE)
+
+install(CODE "
+  message(STATUS \"Install CMake project\")
+")
meta/recipes-devtools/python/python3-cmake/run-cmake-from-path.patch new file +11 -0
new file mode 100644
index 0000000000..5c38fadce0
@@ -0,0 +1,11 @@
+--- cmake-3.25.2/src/cmake/__init__.py.old	2023-03-10 09:40:43.582315753 +0100
++++ cmake-3.25.2/src/cmake/__init__.py	2023-03-10 09:41:38.030874047 +0100
+@@ -36,7 +36,7 @@
+ 
+ 
+ def _program(name, args):
+-    return subprocess.call([os.path.join(CMAKE_BIN_DIR, name)] + args, close_fds=False)
++    return subprocess.call([name] + args, close_fds=False)
+ 
+ 
+ def cmake():
meta/recipes-devtools/python/python3-cmake_3.25.2.bb new file +33 -0
new file mode 100644
index 0000000000..e843293d68
@@ -0,0 +1,33 @@
+SUMMARY = "CMake is an open-source, cross-platform family of tools designed to build, test and package software"
+LICENSE = "BSD-3-Clause & Apache-2.0"
+LIC_FILES_CHKSUM = " \
+	file://LICENSE_BSD_3;md5=9134cb61aebbdd79dd826ccb9ae6afcd \
+	file://LICENSE_Apache_20;md5=19cbd64715b51267a47bf3750cc6a8a5 \
+"
+
+DEPENDS = "ninja-native cmake-native python3-scikit-build-native"
+
+PYPI_PACKAGE = "cmake"
+
+inherit pypi python_setuptools_build_meta
+SRC_URI[sha256sum] = "bcf9f0369743278ec26961542b31ed1610e6f4cfc20c00a3f1c61985abb3b0d2"
+
+SRC_URI += " \
+	file://CMakeLists.txt \
+	file://run-cmake-from-path.patch \
+"
+
+addtask do_patchbuild after do_patch before do_configure
+
+do_patchbuild () {
+	rm -f ${S}/CMakeLists.txt
+	cp ${WORKDIR}/CMakeLists.txt ${S}/
+}
+
+do_install:append () {
+	rm -rf ${D}${bindir}
+}
+
+RDEPENDS:${PN} = "cmake"
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#5 lore raw

PATCH 4/4 python3-ninja: New recipe

In reply to Zoltán Böszörményi

Some python modules rely on "import ninja".

Only build and ship the python parts, and don't download and build ninja from sources. Use the already built ninja instead.

The CMakeLists.txt file is a crippled copy from this ninja python module's sources, removing almost everything, and adding a dummy install target, so do_install() doesn't fail.

The python code is patched so ninja is used from PATH.

---
 .../python/python3-ninja/CMakeLists.txt       |  9 ++++++
 .../python3-ninja/run-ninja-from-path.patch   | 11 +++++++
 .../python/python3-ninja_1.11.1.bb            | 30 +++++++++++++++++++
 3 files changed, 50 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-ninja/CMakeLists.txt
 create mode 100644 meta/recipes-devtools/python/python3-ninja/run-ninja-from-path.patch
 create mode 100644 meta/recipes-devtools/python/python3-ninja_1.11.1.bb
meta/recipes-devtools/python/python3-ninja/CMakeLists.txt new file +9 -0
new file mode 100644
index 0000000000..04fa451e72
@@ -0,0 +1,9 @@
+cmake_minimum_required(VERSION 3.15)
+
+project(NinjaPythonDistributions)
+
+set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_MODULE_PATH})
+
+install(CODE "
+ message(STATUS \"Install ninja project\")
+")
meta/recipes-devtools/python/python3-ninja/run-ninja-from-path.patch new file +11 -0
new file mode 100644
index 0000000000..26bd037373
@@ -0,0 +1,11 @@
+--- ninja-1.11.1/src/ninja/__init__.py.old	2022-11-05 09:49:23.000000000 +0100
++++ ninja-1.11.1/src/ninja/__init__.py	2023-03-10 09:45:13.452082888 +0100
+@@ -44,7 +44,7 @@
+ 
+ 
+ def _program(name, args):
+-    return subprocess.call([os.path.join(BIN_DIR, name)] + args, close_fds=False)
++    return subprocess.call([name] + args, close_fds=False)
+ 
+ 
+ def ninja():
meta/recipes-devtools/python/python3-ninja_1.11.1.bb new file +30 -0
new file mode 100644
index 0000000000..64f5c5c32b
@@ -0,0 +1,30 @@
+SUMMARY = "Ninja is a small build system with a focus on speed"
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM = "file://LICENSE_Apache_20;md5=19cbd64715b51267a47bf3750cc6a8a5"
+
+DEPENDS = "ninja-native cmake-native python3-scikit-build-native"
+
+PYPI_PACKAGE = "ninja"
+
+inherit pypi python_setuptools_build_meta
+SRC_URI[sha256sum] = "c833a47d39b2d1eee3f9ca886fa1581efd5be6068b82734ac229961ee8748f90"
+
+SRC_URI += " \
+	file://CMakeLists.txt \
+	file://run-ninja-from-path.patch \
+"
+
+addtask do_patchbuild after do_patch before do_configure
+
+do_patchbuild () {
+	rm -f ${S}/CMakeLists.txt
+	cp ${WORKDIR}/CMakeLists.txt ${S}/
+}
+
+do_install:append () {
+	rm -rf ${D}${bindir}
+}
+
+RDEPENDS:${PN} = "ninja"
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#6 lore raw
In reply to Zoltán Böszörményi

It would be good to actually test that these new recipes work properly; if they don't have consumers in oe-core, then such tests will not occur. Maybe meta-oe is a better place for now?

Alex

Quoted 14 lines · Zoltan Boszormenyi

Some python modules use "import skbuild", "import cmake" or "import ninja" during their builds.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178315): https://lists.openembedded.org/g/openembedded-core/message/178315
Mute This Topic: https://lists.openembedded.org/mt/97516343/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#7 lore raw

2023. 03. 10. 11:15 keltezéssel, Alexander Kanavin írta:

Quoted 3 lines

It would be good to actually test that these new recipes work properly; if they don't have consumers in oe-core, then such tests will not occur. Maybe meta-oe is a better place for now?

Can you still take patch 1/4 into oe-core to avoid the clash between cmake and python3-cmake if/when the new modules are accepted into meta-oe?

Quoted 17 lines

Alex

Quoted 13 lines · Zoltan Boszormenyi

Some python modules use "import skbuild", "import cmake" or "import ninja" during their builds.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#178315): https://lists.openembedded.org/g/openembedded-core/message/178315
Mute This Topic: https://lists.openembedded.org/mt/97516343/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#8 lore raw
Quoted 9 lines · Böszörményi Zoltán

2023. 03. 10. 11:15 keltezéssel, Alexander Kanavin írta:

Quoted 3 lines

It would be good to actually test that these new recipes work properly; if they don't have consumers in oe-core, then such tests will not occur. Maybe meta-oe is a better place for now?

Can you still take patch 1/4 into oe-core to avoid the clash between cmake and python3-cmake if/when the new modules are accepted into meta-oe?

Of course. Alternatively you can fix the pypi class to allow setting 'extra parameters' in SRC_URI - I don't have a strong opinion, but I think cmake recipe has a stronger claim to the tarball name.

Alex

#9 lore raw

PATCH 1/1 pypi.bbclass: Set SRC_URI downloadfilename with an optional prefix

This allows avoiding clashes between source archives of a main project and a pypi project using the same name and version.

The new optional setting is PYPI_ARCHIVE_NAME_PREFIX which is empty by default so previous downloads can be used. Example usage:

PYPI_ARCHIVE_NAME_PREFIX = "pypi-"

---
 meta/classes-recipe/pypi.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes-recipe/pypi.bbclass +3 -1
index aab04c638f..f510bfed06 100644
@@ -15,11 +15,13 @@ def pypi_package(d):
 PYPI_PACKAGE ?= "${@pypi_package(d)}"
 PYPI_PACKAGE_EXT ?= "tar.gz"
 PYPI_ARCHIVE_NAME ?= "${PYPI_PACKAGE}-${PV}.${PYPI_PACKAGE_EXT}"
+PYPI_ARCHIVE_NAME_PREFIX ?= ""
 
 def pypi_src_uri(d):
     package = d.getVar('PYPI_PACKAGE')
     archive_name = d.getVar('PYPI_ARCHIVE_NAME')
-    return 'https://files.pythonhosted.org/packages/source/%s/%s/%s' % (package[0], package, archive_name)
+    archive_downloadname = d.getVar('PYPI_ARCHIVE_NAME_PREFIX') + archive_name
+    return 'https://files.pythonhosted.org/packages/source/%s/%s/%s;downloadfilename=%s' % (package[0], package, archive_name, archive_downloadname)
 
 PYPI_SRC_URI ?= "${@pypi_src_uri(d)}"
 
Signature · 1 line
2.39.2
#10 lore raw

Re: [PATCH] pypi.bbclass: Set SRC_URI downloadfilename with an optional prefix

Thanks, I like this better than changing the cmake recipe.

Alex

Quoted 36 lines · Zoltán Böszörményi

This allows avoiding clashes between source archives of a main project and a pypi project using the same name and version.

The new optional setting is PYPI_ARCHIVE_NAME_PREFIX which is empty by default so previous downloads can be used. Example usage:

PYPI_ARCHIVE_NAME_PREFIX = "pypi-"

---
 meta/classes-recipe/pypi.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes-recipe/pypi.bbclass +3 -2
index aab04c638f..f510bfed06 100644
@@ -15,11 +15,13 @@ def pypi_package(d):
 PYPI_PACKAGE ?= "${@pypi_package(d)}"
 PYPI_PACKAGE_EXT ?= "tar.gz"
 PYPI_ARCHIVE_NAME ?= "${PYPI_PACKAGE}-${PV}.${PYPI_PACKAGE_EXT}"
+PYPI_ARCHIVE_NAME_PREFIX ?= ""
 
 def pypi_src_uri(d):
     package = d.getVar('PYPI_PACKAGE')
     archive_name = d.getVar('PYPI_ARCHIVE_NAME')
-    return 'https://files.pythonhosted.org/packages/source/%s/%s/%s' % (package[0], package, archive_name)
+    archive_downloadname = d.getVar('PYPI_ARCHIVE_NAME_PREFIX') + archive_name
+    return 'https://files.pythonhosted.org/packages/source/%s/%s/%s;downloadfilename=%s' % (package[0], package, archive_name, archive_downloadname)
 
 PYPI_SRC_URI ?= "${@pypi_src_uri(d)}"
 
--

2.39.2

#11 lore raw

Re: [PATCH] pypi.bbclass: Set SRC_URI downloadfilename with an optional prefix

2023. 03. 10. 13:45 keltezéssel, Alexander Kanavin írta:

Quoted 1 line

Thanks, I like this better than changing the cmake recipe.

Thank you. I will resend the new module recipes against meta-oe soon.

Quoted 39 lines

Alex

Quoted 35 lines · Zoltán Böszörményi

This allows avoiding clashes between source archives of a main project and a pypi project using the same name and version.

The new optional setting is PYPI_ARCHIVE_NAME_PREFIX which is empty by default so previous downloads can be used. Example usage:

PYPI_ARCHIVE_NAME_PREFIX = "pypi-"

---
  meta/classes-recipe/pypi.bbclass | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes-recipe/pypi.bbclass +3 -2
index aab04c638f..f510bfed06 100644
@@ -15,11 +15,13 @@ def pypi_package(d):
  PYPI_PACKAGE ?= "${@pypi_package(d)}"
  PYPI_PACKAGE_EXT ?= "tar.gz"
  PYPI_ARCHIVE_NAME ?= "${PYPI_PACKAGE}-${PV}.${PYPI_PACKAGE_EXT}"
+PYPI_ARCHIVE_NAME_PREFIX ?= ""
 
  def pypi_src_uri(d):
      package = d.getVar('PYPI_PACKAGE')
      archive_name = d.getVar('PYPI_ARCHIVE_NAME')
-    return 'https://files.pythonhosted.org/packages/source/%s/%s/%s' % (package[0], package, archive_name)
+    archive_downloadname = d.getVar('PYPI_ARCHIVE_NAME_PREFIX') + archive_name
+    return 'https://files.pythonhosted.org/packages/source/%s/%s/%s;downloadfilename=%s' % (package[0], package, archive_name, archive_downloadname)
 
  PYPI_SRC_URI ?= "${@pypi_src_uri(d)}"
 
--

2.39.2