Skip to contentSkip to conversation

package/python-maturin: new package

PATCH Superseded James Hilliard 3 participants 6 messages May 10, 2022 – May 11, 2022

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 package/python-maturin: new package

This is pep517 pyo3 build backend that's an alternative to python-setuptools-rust.

---
Changes v1 -> v2:
- update patch comment to backport as it is merged upstream

---
 .../0001-Fix-invalid-pyproject.toml.patch     | 62 +++++++++++++++++++
 package/python-maturin/python-maturin.hash    |  5 ++
 package/python-maturin/python-maturin.mk      | 20 ++++++
 3 files changed, 87 insertions(+)
 create mode 100644 package/python-maturin/0001-Fix-invalid-pyproject.toml.patch
 create mode 100644 package/python-maturin/python-maturin.hash
 create mode 100644 package/python-maturin/python-maturin.mk
package/python-maturin/0001-Fix-invalid-pyproject.toml.patch new file +62 -0
new file mode 100644
index 0000000000..91e9a3db94
@@ -0,0 +1,62 @@
+From 9ad69941d02e494d45ee67f3bcccc83c0bb4cdcd Mon Sep 17 00:00:00 2001
+From: James Hilliard <[email protected]>
+Date: Mon, 9 May 2022 14:37:36 -0600
+Subject: [PATCH] Fix invalid pyproject.toml
+
+These attributes should be marked dynamic.
+
+Fixes:
+configuration error: `project` must contain ['version'] properties
+
+Fixes:
+The following seems to be defined outside of `pyproject.toml`:
+
+`description = 'Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packages'`
+
+According to the spec (see the link bellow), however, setuptools CANNOT
+consider this value unless 'description' is listed as `dynamic`.
+
+Fixes:
+The following seems to be defined outside of `pyproject.toml`:
+
+`license = 'MIT OR Apache-2.0'`
+
+According to the spec (see the link bellow), however, setuptools CANNOT
+consider this value unless 'license' is listed as `dynamic`.
+
+Fixes:
+The following seems to be defined outside of `pyproject.toml`:
+
+`authors = 'konstin'`
+
+According to the spec (see the link bellow), however, setuptools CANNOT
+consider this value unless 'authors' is listed as `dynamic`.
+
+Signed-off-by: James Hilliard <[email protected]>
+[[email protected]: backport from upstream commit
+9ad69941d02e494d45ee67f3bcccc83c0bb4cdcd]
+---
+ pyproject.toml | 7 +++++++
+ 1 file changed, 7 insertions(+)
+
+diff --git a/pyproject.toml b/pyproject.toml
+index 207ceb7..329faae 100644
+--- a/pyproject.toml
++++ b/pyproject.toml
+@@ -13,6 +13,13 @@ classifiers = [
+     "Programming Language :: Python :: Implementation :: PyPy",
+ ]
+ dependencies = ["tomli>=1.1.0 ; python_version<'3.11'"]
++dynamic = [
++    "authors",
++    "description",
++    "license",
++    "readme",
++    "version"
++]
+ 
+ [project.optional-dependencies]
+ zig = [
+-- 
+2.25.1
+
package/python-maturin/python-maturin.hash new file +5 -0
new file mode 100644
index 0000000000..e9ba83e78e
@@ -0,0 +1,5 @@
+# Locally calculated after vendoring
+sha256  092db51dfd1479e3abfc8eef4b109b6ab504481959e383090827e2024f09aab3  maturin-0.12.15.tar.gz
+# Locally computed sha256 checksums
+sha256  a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2  license-apache
+sha256  ea7882c559733766ad08343bde1d1ec80a4967c03a738fb8e0058ef6289f7b7c  license-mit
package/python-maturin/python-maturin.mk new file +20 -0
new file mode 100644
index 0000000000..f6daabc008
@@ -0,0 +1,20 @@
+################################################################################
+#
+# python-maturin
+#
+################################################################################
+
+PYTHON_MATURIN_VERSION = 0.12.15
+PYTHON_MATURIN_SOURCE = maturin-$(PYTHON_MATURIN_VERSION).tar.gz
+PYTHON_MATURIN_SITE = https://files.pythonhosted.org/packages/30/4d/b8a32e0bd711f7905fa8f10487be7dbf9992c5b5b3bfa8734abf2487d967
+PYTHON_MATURIN_SETUP_TYPE = setuptools
+PYTHON_MATURIN_LICENSE = Apache-2.0 or MIT
+PYTHON_MATURIN_LICENSE_FILES = license-apache license-mit
+HOST_PYTHON_MATURIN_DEPENDENCIES = host-python-tomli host-rustc
+HOST_PYTHON_MATURIN_ENV = $(HOST_PKG_CARGO_ENV)
+# We need to vendor the Cargo crates at download time
+PYTHON_MATURIN_DOWNLOAD_POST_PROCESS = cargo
+PYTHON_MATURIN_DOWNLOAD_DEPENDENCIES = host-rustc
+HOST_PYTHON_MATURIN_DL_ENV = $(HOST_PKG_CARGO_ENV)
+
+$(eval $(host-python-package))
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH v2 2/2 package/python-orjson: bump to version 3.6.8

Convert package to use python infrastructure instead of cargo infrastructure.

---
 package/python-orjson/python-orjson.hash |  5 +++--
 package/python-orjson/python-orjson.mk   | 24 ++++++++++++------------
 2 files changed, 15 insertions(+), 14 deletions(-)
package/python-orjson/python-orjson.hash +3 -2
index 5ef2726d82..06cc5cae79 100644
@@ -1,4 +1,5 @@
-# Locally calculated
-sha256  4713b120ad86b1b273f5952cbf9578c13d0f453064bd26d823e04a10c84824e6  python-orjson-3.6.7.tar.gz
+# Locally calculated after vendoring
+sha256  40b95576b81916d167f7a44829401acb3b6cbbfe9840b5ed88c9535a86507d7b  orjson-3.6.8.tar.gz
+# Locally computed sha256 checksums
 sha256  a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2  LICENSE-APACHE
 sha256  23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3  LICENSE-MIT
package/python-orjson/python-orjson.mk +12 -12
index 5de4b2c5fa..ccbaf37c6d 100644
@@ -4,19 +4,19 @@
 #
 ################################################################################
 
-PYTHON_ORJSON_VERSION = 3.6.7
-PYTHON_ORJSON_SITE = $(call github,ijl,orjson,$(PYTHON_ORJSON_VERSION))
+PYTHON_ORJSON_VERSION = 3.6.8
+PYTHON_ORJSON_SOURCE = orjson-$(PYTHON_ORJSON_VERSION).tar.gz
+PYTHON_ORJSON_SITE = https://files.pythonhosted.org/packages/14/8e/7e745d867466bdafb12ab7b0179d40f23f55639f7cd14bd375acfa74aca2
+PYTHON_ORJSON_SETUP_TYPE = pep517
 PYTHON_ORJSON_LICENSE = Apache-2.0 or MIT
 PYTHON_ORJSON_LICENSE_FILES = LICENSE-APACHE LICENSE-MIT
-PYTHON_ORJSON_DEPENDENCIES = host-python-cffi
-PYTHON_ORJSON_CARGO_ENV = \
+PYTHON_ORJSON_DEPENDENCIES = host-python-cffi host-python-maturin
+PYTHON_ORJSON_ENV = \
+	$(PKG_CARGO_ENV) \
 	PYO3_CROSS_LIB_DIR="$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)"
+# We need to vendor the Cargo crates at download time
+PYTHON_ORJSON_DOWNLOAD_POST_PROCESS = cargo
+PYTHON_ORJSON_DOWNLOAD_DEPENDENCIES = host-rustc
+PYTHON_ORJSON_DL_ENV = $(PKG_CARGO_ENV)
 
-# orjson uses "maturin" to generate distribution packages - rather than teach
-# buildroot how to understand this, we reach in and install directly.
-define PYTHON_ORJSON_INSTALL_TARGET_CMDS
-	$(INSTALL) -m 0755 -D $(@D)/target/$(RUSTC_TARGET_NAME)/release/liborjson.so \
-		$(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/orjson.so
-endef
-
-$(eval $(cargo-package))
+$(eval $(python-package))
Signature · 6 lines
2.25.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [PATCH v2 2/2] package/python-orjson: bump to version 3.6.8

Quoted 2 lines · James Hilliard

Convert package to use python infrastructure instead of cargo infrastructure.

AFAIU, maturin was already available in 3.6.7, so the conversion should be
separate from the bump.

That said, I think Graeme intentionally did not use maturin because it doesn't
really offer much benefit over calling cargo directly. Do you have a reason to prefer using maturin?

Quoted 24 lines
---
  package/python-orjson/python-orjson.hash |  5 +++--
  package/python-orjson/python-orjson.mk   | 24 ++++++++++++------------
  2 files changed, 15 insertions(+), 14 deletions(-)
package/python-orjson/python-orjson.hash +3 -2
index 5ef2726d82..06cc5cae79 100644
@@ -1,4 +1,5 @@
-# Locally calculated
-sha256  4713b120ad86b1b273f5952cbf9578c13d0f453064bd26d823e04a10c84824e6  python-orjson-3.6.7.tar.gz
+# Locally calculated after vendoring
+sha256  40b95576b81916d167f7a44829401acb3b6cbbfe9840b5ed88c9535a86507d7b  orjson-3.6.8.tar.gz
+# Locally computed sha256 checksums
  sha256  a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2  LICENSE-APACHE
  sha256  23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3  LICENSE-MIT
package/python-orjson/python-orjson.mk +0 -0
index 5de4b2c5fa..ccbaf37c6d 100644

package/python-orjson/python-orjson.mk line 9

Quoted 8 lines
package/python-orjson/python-orjson.mk +3 -2
@@ -4,19 +4,19 @@
  #
  ################################################################################
 
-PYTHON_ORJSON_VERSION = 3.6.7
-PYTHON_ORJSON_SITE = $(call github,ijl,orjson,$(PYTHON_ORJSON_VERSION))
+PYTHON_ORJSON_VERSION = 3.6.8
+PYTHON_ORJSON_SOURCE = orjson-$(PYTHON_ORJSON_VERSION).tar.gz
+PYTHON_ORJSON_SITE = https://files.pythonhosted.org/packages/14/8e/7e745d867466bdafb12ab7b0179d40f23f55639f7cd14bd375acfa74aca2

Kind of an aside, since this applies to many other packages: is there a reason
we prefer to get Python packages from pypi rather than the upstream source itself?

package/python-orjson/python-orjson.mk line 20

Quoted 13 lines
package/python-orjson/python-orjson.mk +8 -2
+PYTHON_ORJSON_SETUP_TYPE = pep517
  PYTHON_ORJSON_LICENSE = Apache-2.0 or MIT
  PYTHON_ORJSON_LICENSE_FILES = LICENSE-APACHE LICENSE-MIT
-PYTHON_ORJSON_DEPENDENCIES = host-python-cffi
-PYTHON_ORJSON_CARGO_ENV = \
+PYTHON_ORJSON_DEPENDENCIES = host-python-cffi host-python-maturin
+PYTHON_ORJSON_ENV = \
+	$(PKG_CARGO_ENV) \
  	PYO3_CROSS_LIB_DIR="$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)"
+# We need to vendor the Cargo crates at download time
+PYTHON_ORJSON_DOWNLOAD_POST_PROCESS = cargo
+PYTHON_ORJSON_DOWNLOAD_DEPENDENCIES = host-rustc
+PYTHON_ORJSON_DL_ENV = $(PKG_CARGO_ENV)

Is this going to be needed for all maturin packages? If yes, it will make
sense to move this into a maturin setup type (but preferably only after we have at least 3 packages that use it).

Regards,
Arnout

package/python-orjson/python-orjson.mk line 22

Quoted 10 lines
package/python-orjson/python-orjson.mk +1 -8
-# orjson uses "maturin" to generate distribution packages - rather than teach
-# buildroot how to understand this, we reach in and install directly.
-define PYTHON_ORJSON_INSTALL_TARGET_CMDS
-	$(INSTALL) -m 0755 -D $(@D)/target/$(RUSTC_TARGET_NAME)/release/liborjson.so \
-		$(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/orjson.so
-endef
-
-$(eval $(cargo-package))
+$(eval $(python-package))

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw

Re: [Buildroot] [PATCH v2 2/2] package/python-orjson: bump to version 3.6.8

Quoted 9 lines · Arnout Vandecappelle
Quoted 2 lines · James Hilliard

Convert package to use python infrastructure instead of cargo infrastructure.

AFAIU, maturin was already available in 3.6.7, so the conversion should be
separate from the bump.

I mostly did it at the same time to avoid any potential conflicts/confusion with mirrored assets.

Quoted 4 lines

That said, I think Graeme intentionally did not use maturin because it doesn't
really offer much benefit over calling cargo directly. Do you have a reason to prefer using maturin?

It should be more correct, I mean calling cargo directly doesn't really properly install say pure python files and is more likely to break in weird ways.

Quoted 28 lines
Quoted 28 lines
---
  package/python-orjson/python-orjson.hash |  5 +++--
  package/python-orjson/python-orjson.mk   | 24 ++++++++++++------------
  2 files changed, 15 insertions(+), 14 deletions(-)
package/python-orjson/python-orjson.hash +3 -2
index 5ef2726d82..06cc5cae79 100644
@@ -1,4 +1,5 @@
-# Locally calculated
-sha256  4713b120ad86b1b273f5952cbf9578c13d0f453064bd26d823e04a10c84824e6  python-orjson-3.6.7.tar.gz
+# Locally calculated after vendoring
+sha256  40b95576b81916d167f7a44829401acb3b6cbbfe9840b5ed88c9535a86507d7b  orjson-3.6.8.tar.gz
+# Locally computed sha256 checksums
  sha256  a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2  LICENSE-APACHE
  sha256  23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3  LICENSE-MIT
package/python-orjson/python-orjson.mk +0 -0
index 5de4b2c5fa..ccbaf37c6d 100644
@@ -4,19 +4,19 @@
  #
  ################################################################################
 
Quoted 8 lines

package/python-orjson/python-orjson.mk line 9

Quoted 5 lines
package/python-orjson/python-orjson.mk +3 -2
-PYTHON_ORJSON_VERSION = 3.6.7
-PYTHON_ORJSON_SITE = $(call github,ijl,orjson,$(PYTHON_ORJSON_VERSION))
+PYTHON_ORJSON_VERSION = 3.6.8
+PYTHON_ORJSON_SOURCE = orjson-$(PYTHON_ORJSON_VERSION).tar.gz
+PYTHON_ORJSON_SITE = https://files.pythonhosted.org/packages/14/8e/7e745d867466bdafb12ab7b0179d40f23f55639f7cd14bd375acfa74aca2

Kind of an aside, since this applies to many other packages: is there a reason
we prefer to get Python packages from pypi rather than the upstream source itself?

These are sdist's(release tarball equivalents for python packages) while upstream
source is not.

Quoted 10 lines
Quoted 10 lines
package/python-orjson/python-orjson.mk +4 -2
+PYTHON_ORJSON_SETUP_TYPE = pep517
  PYTHON_ORJSON_LICENSE = Apache-2.0 or MIT
  PYTHON_ORJSON_LICENSE_FILES = LICENSE-APACHE LICENSE-MIT
-PYTHON_ORJSON_DEPENDENCIES = host-python-cffi
-PYTHON_ORJSON_CARGO_ENV = \
+PYTHON_ORJSON_DEPENDENCIES = host-python-cffi host-python-maturin
+PYTHON_ORJSON_ENV = \
+     $(PKG_CARGO_ENV) \
      PYO3_CROSS_LIB_DIR="$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)"
Quoted 8 lines

package/python-orjson/python-orjson.mk line 20

Quoted 4 lines
package/python-orjson/python-orjson.mk +4 -0
+# We need to vendor the Cargo crates at download time
+PYTHON_ORJSON_DOWNLOAD_POST_PROCESS = cargo
+PYTHON_ORJSON_DOWNLOAD_DEPENDENCIES = host-rustc
+PYTHON_ORJSON_DL_ENV = $(PKG_CARGO_ENV)

Is this going to be needed for all maturin packages? If yes, it will make
sense to move this into a maturin setup type (but preferably only after we have at least 3 packages that use it).

Yes, I'm planning on adding that along with a setuptools-rust setup type, I have some upstream work that should making things clearer as well like this:
https://github.com/PyO3/maturin/pull/909

I'm going to rework this to handle maturin as well(both are pyo3 build tools):
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Quoted 5 lines

Regards,
Arnout

Quoted 1 line
Quoted 9 lines

package/python-orjson/python-orjson.mk line 22

Quoted 9 lines
package/python-orjson/python-orjson.mk +1 -8
-# orjson uses "maturin" to generate distribution packages - rather than teach
-# buildroot how to understand this, we reach in and install directly.
-define PYTHON_ORJSON_INSTALL_TARGET_CMDS
-     $(INSTALL) -m 0755 -D $(@D)/target/$(RUSTC_TARGET_NAME)/release/liborjson.so \
-             $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/orjson.so
-endef
-
-$(eval $(cargo-package))
+$(eval $(python-package))

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#4 lore raw

Re: [Buildroot] [PATCH v2 2/2] package/python-orjson: bump to version 3.6.8

Hi James and Arnout,

Quoted 15 lines · James Hilliard
Quoted 9 lines · Arnout Vandecappelle
Quoted 2 lines · James Hilliard

Convert package to use python infrastructure instead of cargo infrastructure.

AFAIU, maturin was already available in 3.6.7, so the conversion should be
separate from the bump.

I mostly did it at the same time to avoid any potential conflicts/confusion with mirrored assets.

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

That said, I think Graeme intentionally did not use maturin because it doesn't
really offer much benefit over calling cargo directly. Do you have a reason to prefer using maturin?

It should be more correct, I mean calling cargo directly doesn't really properly install say pure python files and is more likely to break in weird ways.

The python-maturin package wasn't available when I initially packaged orjson (3.6.7).

I suspect if this patch works, then shifting python-orjson to python-maturin will be less brittle (or, brittle in ways that are less isolated) than the current maturin bypass.

If the extra information would be helpful, I can give this python-maturin approach a try and see if it works in my application.

best,
Graeme

Quoted 73 lines
Quoted 29 lines
Quoted 29 lines
---
   package/python-orjson/python-orjson.hash |  5 +++--
   package/python-orjson/python-orjson.mk   | 24 ++++++++++++------------
   2 files changed, 15 insertions(+), 14 deletions(-)
package/python-orjson/python-orjson.hash +3 -2
index 5ef2726d82..06cc5cae79 100644
@@ -1,4 +1,5 @@
-# Locally calculated
-sha256  4713b120ad86b1b273f5952cbf9578c13d0f453064bd26d823e04a10c84824e6  python-orjson-3.6.7.tar.gz
+# Locally calculated after vendoring
+sha256  40b95576b81916d167f7a44829401acb3b6cbbfe9840b5ed88c9535a86507d7b  orjson-3.6.8.tar.gz
+# Locally computed sha256 checksums
   sha256  a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2  LICENSE-APACHE
   sha256  23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3  LICENSE-MIT
package/python-orjson/python-orjson.mk +0 -0
index 5de4b2c5fa..ccbaf37c6d 100644
@@ -4,19 +4,19 @@
   #
   ################################################################################
 
Quoted 8 lines

package/python-orjson/python-orjson.mk line 9

Quoted 5 lines
package/python-orjson/python-orjson.mk +3 -2
-PYTHON_ORJSON_VERSION = 3.6.7
-PYTHON_ORJSON_SITE = $(call github,ijl,orjson,$(PYTHON_ORJSON_VERSION))
+PYTHON_ORJSON_VERSION = 3.6.8
+PYTHON_ORJSON_SOURCE = orjson-$(PYTHON_ORJSON_VERSION).tar.gz
+PYTHON_ORJSON_SITE = https://files.pythonhosted.org/packages/14/8e/7e745d867466bdafb12ab7b0179d40f23f55639f7cd14bd375acfa74aca2

Kind of an aside, since this applies to many other packages: is there a reason
we prefer to get Python packages from pypi rather than the upstream source itself?

These are sdist's(release tarball equivalents for python packages) while upstream
source is not.

Quoted 10 lines
Quoted 10 lines
package/python-orjson/python-orjson.mk +4 -2
+PYTHON_ORJSON_SETUP_TYPE = pep517
   PYTHON_ORJSON_LICENSE = Apache-2.0 or MIT
   PYTHON_ORJSON_LICENSE_FILES = LICENSE-APACHE LICENSE-MIT
-PYTHON_ORJSON_DEPENDENCIES = host-python-cffi
-PYTHON_ORJSON_CARGO_ENV = \
+PYTHON_ORJSON_DEPENDENCIES = host-python-cffi host-python-maturin
+PYTHON_ORJSON_ENV = \
+     $(PKG_CARGO_ENV) \
       PYO3_CROSS_LIB_DIR="$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)"
Quoted 8 lines

package/python-orjson/python-orjson.mk line 20

Quoted 4 lines
package/python-orjson/python-orjson.mk +4 -0
+# We need to vendor the Cargo crates at download time
+PYTHON_ORJSON_DOWNLOAD_POST_PROCESS = cargo
+PYTHON_ORJSON_DOWNLOAD_DEPENDENCIES = host-rustc
+PYTHON_ORJSON_DL_ENV = $(PKG_CARGO_ENV)

Is this going to be needed for all maturin packages? If yes, it will make
sense to move this into a maturin setup type (but preferably only after we have at least 3 packages that use it).

Yes, I'm planning on adding that along with a setuptools-rust setup type, I have some upstream work that should making things clearer as well like this:
https://github.com/PyO3/maturin/pull/909

I'm going to rework this to handle maturin as well(both are pyo3 build tools):
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Quoted 5 lines

Regards,
Arnout

Quoted 1 line
Quoted 9 lines
Quoted 9 lines

package/python-orjson/python-orjson.mk line 22

Quoted 9 lines
package/python-orjson/python-orjson.mk +1 -8
-# orjson uses "maturin" to generate distribution packages - rather than teach
-# buildroot how to understand this, we reach in and install directly.
-define PYTHON_ORJSON_INSTALL_TARGET_CMDS
-     $(INSTALL) -m 0755 -D $(@D)/target/$(RUSTC_TARGET_NAME)/release/liborjson.so \
-             $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/orjson.so
-endef
-
-$(eval $(cargo-package))
+$(eval $(python-package))

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#5 lore raw

Re: [Buildroot] [PATCH v2 2/2] package/python-orjson: bump to version 3.6.8

Quoted 22 lines · Graeme Smecher

Hi James and Arnout,

Quoted 19 lines · James Hilliard
Quoted 9 lines · Arnout Vandecappelle
Quoted 2 lines · James Hilliard

Convert package to use python infrastructure instead of cargo infrastructure.

   AFAIU, maturin was already available in 3.6.7, so the conversion should be separate from the bump.

I mostly did it at the same time to avoid any potential conflicts/confusion with mirrored assets.

Quoted 4 lines

   That said, I think Graeme intentionally did not use maturin because it doesn't really offer much benefit over calling cargo directly. Do you have a reason to prefer using maturin?

Quoted 8 lines
Quoted 2 lines

It should be more correct, I mean calling cargo directly doesn't really properly install say pure python files and is more likely to break in weird ways.

The python-maturin package wasn't available when I initially packaged orjson (3.6.7).

I suspect if this patch works, then shifting python-orjson to python-maturin will be less brittle (or, brittle in ways that are less isolated) than the current maturin bypass.

If the extra information would be helpful, I can give this python-maturin approach a try and see if it works in my application.

Yes please! If it's good, then please add your Tested-by to the first patch in
this series.

Regards,
Arnout

Quoted 85 lines

best,
Graeme

Quoted 81 lines
Quoted 33 lines
Quoted 33 lines

---
   package/python-orjson/python-orjson.hash |  5 +++-- package/python-orjson/python-orjson.mk   | 24 ++++++++++++------------

   2 files changed, 15 insertions(+), 14 deletions(-)
package/python-orjson/python-orjson.hash +0 -2
index 5ef2726d82..06cc5cae79 100644
@@ -1,4 +1,5 @@
-# Locally calculated
-sha256  4713b120ad86b1b273f5952cbf9578c13d0f453064bd26d823e04a10c84824e6  

python-orjson-3.6.7.tar.gz

package/python-orjson/python-orjson.hash +2 -0
+# Locally calculated after vendoring
+sha256  40b95576b81916d167f7a44829401acb3b6cbbfe9840b5ed88c9535a86507d7b  

orjson-3.6.8.tar.gz
+# Locally computed sha256 checksums
   sha256  a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 LICENSE-APACHE sha256  23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 LICENSE-MIT

package/python-orjson/python-orjson.mk +0 -0
index 5de4b2c5fa..ccbaf37c6d 100644
@@ -4,19 +4,19 @@
    #
 
 ################################################################################ 
 
 
Quoted 10 lines
Quoted 7 lines
package/python-orjson/python-orjson.mk +3 -2
-PYTHON_ORJSON_VERSION = 3.6.7
-PYTHON_ORJSON_SITE = $(call github,ijl,orjson,$(PYTHON_ORJSON_VERSION))
+PYTHON_ORJSON_VERSION = 3.6.8
+PYTHON_ORJSON_SOURCE = orjson-$(PYTHON_ORJSON_VERSION).tar.gz
+PYTHON_ORJSON_SITE = 

https://files.pythonhosted.org/packages/14/8e/7e745d867466bdafb12ab7b0179d40f23f55639f7cd14bd375acfa74aca2

   Kind of an aside, since this applies to many other packages: is there a reason we prefer to get Python packages from pypi rather than the upstream source itself?

These are sdist's(release tarball equivalents for python packages) while upstream
source is not.

Quoted 10 lines
Quoted 10 lines

+PYTHON_ORJSON_SETUP_TYPE = pep517 PYTHON_ORJSON_LICENSE = Apache-2.0 or MIT PYTHON_ORJSON_LICENSE_FILES = LICENSE-APACHE LICENSE-MIT

package/python-orjson/python-orjson.mk +3 -2
-PYTHON_ORJSON_DEPENDENCIES = host-python-cffi
-PYTHON_ORJSON_CARGO_ENV = \
+PYTHON_ORJSON_DEPENDENCIES = host-python-cffi host-python-maturin
+PYTHON_ORJSON_ENV = \
+     $(PKG_CARGO_ENV) \
Quoted 9 lines

package/python-orjson/python-orjson.mk line 20

Quoted 5 lines

PYO3_CROSS_LIB_DIR="$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)"

package/python-orjson/python-orjson.mk +4 -0
+# We need to vendor the Cargo crates at download time
+PYTHON_ORJSON_DOWNLOAD_POST_PROCESS = cargo
+PYTHON_ORJSON_DOWNLOAD_DEPENDENCIES = host-rustc
+PYTHON_ORJSON_DL_ENV = $(PKG_CARGO_ENV)

   Is this going to be needed for all maturin packages? If yes, it will make sense to move this into a maturin setup type (but preferably only after we have at least 3 packages that use it).

Yes, I'm planning on adding that along with a setuptools-rust setup type, I have some upstream work that should making things clearer as well like this:
https://github.com/PyO3/maturin/pull/909

I'm going to rework this to handle maturin as well(both are pyo3 build tools):
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Quoted 5 lines

   Regards,
   Arnout

Quoted 1 line
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

package/python-orjson/python-orjson.mk line 22

Quoted 11 lines
package/python-orjson/python-orjson.mk +0 -4
-# orjson uses "maturin" to generate distribution packages - rather than teach
-# buildroot how to understand this, we reach in and install directly.
-define PYTHON_ORJSON_INSTALL_TARGET_CMDS
-     $(INSTALL) -m 0755 -D 

$(@D)/target/$(RUSTC_TARGET_NAME)/release/liborjson.so \
- $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/orjson.so

package/python-orjson/python-orjson.mk +1 -3
-endef
-
-$(eval $(cargo-package))
+$(eval $(python-package))

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot