Skip to contentSkip to conversation

python3: fix run-time dependencies

PATCH In review Bartosz Golaszewski 7 participants 30 messages May 17, 2023 – May 19, 2023

lore
Share & tools
lore
#0 lore raw

cover python3: fix run-time dependencies

From: Bartosz Golaszewski <[email protected]>

I noticed that every now and then I import some python package into an image only to find out the recipe didn't pull in all required run-time dependencies. I would fix it whenever I noticed it but figured that I could right a script that would go over all python packages, build a minimal image and see that all its modules can be imported.

The patches in this series are the result of that. Most simply add missing RDEPENDS, some are additional tweaks to recipes and the last three add new standard library packages to the manifest. These are the ones for which simply adding python3-misc would still be insufficient in terms of run-time dependencies.

I plan to do the same for meta-python but this will take more time and I need to think of some automated way of updating the recipes first.

Bartosz Golaszewski (20):
python3-async: add missing run-time dependencies
python3-attrs: unify RDEPENDS
python3-attrs: don't use PYTHON_PN
python3-attrs: add missing run-time dependencies
python3-certifi: add missing run-time dependencies
python3-libfdt: new package
python3-dtschema: add missing run-time dependencies
python3-hypothesis: fix run-time dependencies
python3-tomli: add missing run-time dependencies
python3-pathspec: add missing run-time dependencies
python3-installer: add missing run-time dependencies
python3-sphinx-rtd-theme: add missing run-time dependencies
python3-setuptools-rust: fix RDEPENDS and allow target build
python3-pyproject-hooks: add missing run-time dependencies
python3-pycryptodome: don't use PYTHON_PN
python3-pycryptodome: add missing run-time dependencies
python3-pygobject: add missing run-time dependencies
python3-manifest: cgitb: new package
python3-manifest: zipapp: new package
python3-manifest: turtle: new package

 meta/recipes-devtools/python/python-async.inc |  5 ++-
 .../python/python-pycryptodome.inc            |  8 ++--
 .../python/python3-attrs_23.1.0.bb            | 15 +++----
 .../python/python3-certifi_2022.12.7.bb       |  2 +
 .../python/python3-dtschema_2023.4.bb         |  7 +++-
 .../python/python3-hypothesis_6.71.0.bb       |  2 +-
 .../python/python3-installer_0.7.0.bb         |  6 +++
 .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++
 .../python/python3-pathspec_0.11.1.bb         |  2 +
 .../python/python3-pygobject_3.44.1.bb        |  5 ++-
 .../python/python3-pyproject-hooks_1.0.0.bb   |  5 ++-
 .../python/python3-setuptools-rust_1.5.2.bb   | 17 ++++----
 .../python/python3-sphinx-rtd-theme_1.2.0.bb  |  5 ++-
 .../python/python3-tomli_2.0.1.bb             |  5 +++
 .../python/python3/python3-manifest.json      | 42 +++++++++++++++++++
 15 files changed, 121 insertions(+), 25 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb
Signature · 1 line
2.39.2
#1 lore raw

PATCH 1/20 python3-async: add missing run-time dependencies

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python-async.inc | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python-async.inc +4 -1
index fde864601c..53f8d79361 100644
@@ -9,6 +9,9 @@ inherit pypi
 SRC_URI[md5sum] = "9b06b5997de2154f3bc0273f80bcef6b"
 SRC_URI[sha256sum] = "ac6894d876e45878faae493b0cf61d0e28ec417334448ac0a6ea2229d8343051"
 
-RDEPENDS:${PN} += "${PYTHON_PN}-threading"
+RDEPENDS:${PN} += " \
+    python3-logging \
+    python3-threading \
+"
 
 BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#2 lore raw

Re: [OE-core][PATCH 01/20] python3-async: add missing run-time dependencies

Quoted 14 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
  meta/recipes-devtools/python/python-async.inc | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python-async.inc +0 -0
index fde864601c..53f8d79361 100644

meta/recipes-devtools/python/python-async.inc inherit pypi · line 17

Quoted 10 lines
meta/recipes-devtools/python/python-async.inc +4 -1
@@ -9,6 +9,9 @@ inherit pypi
  SRC_URI[md5sum] = "9b06b5997de2154f3bc0273f80bcef6b"
  SRC_URI[sha256sum] = "ac6894d876e45878faae493b0cf61d0e28ec417334448ac0a6ea2229d8343051"
 
-RDEPENDS:${PN} += "${PYTHON_PN}-threading"
+RDEPENDS:${PN} += " \
+    python3-logging \
+    python3-threading \
+"
 
  BBCLASSEXTEND = "native nativesdk"

This is an OK change, but the recipe should arguably be removed from oe-core instead, given that it 1) hasn't been updated in 9 years and has been abandoned in favor of alternatives by the maintainer, and 2) doesn't appear to be a dependency for anything else in oe-core or meta-openembedded.

- Trevor

Quoted 9 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181450): https://lists.openembedded.org/g/openembedded-core/message/181450
Mute This Topic: https://lists.openembedded.org/mt/98943997/7611679
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Re: [OE-core][PATCH 01/20] python3-async: add missing run-time dependencies

Quoted 34 lines · Trevor Gamblin
Quoted 14 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
  meta/recipes-devtools/python/python-async.inc | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python-async.inc +0 -0
index fde864601c..53f8d79361 100644

meta/recipes-devtools/python/python-async.inc inherit pypi · line 17

Quoted 10 lines
meta/recipes-devtools/python/python-async.inc +4 -1
@@ -9,6 +9,9 @@ inherit pypi
  SRC_URI[md5sum] = "9b06b5997de2154f3bc0273f80bcef6b"
  SRC_URI[sha256sum] = "ac6894d876e45878faae493b0cf61d0e28ec417334448ac0a6ea2229d8343051"
 
-RDEPENDS:${PN} += "${PYTHON_PN}-threading"
+RDEPENDS:${PN} += " \
+    python3-logging \
+    python3-threading \
+"
 
  BBCLASSEXTEND = "native nativesdk"

This is an OK change, but the recipe should arguably be removed from oe-core instead, given that it 1) hasn't been updated in 9 years and has been abandoned in favor of alternatives by the maintainer, and 2) doesn't appear to be a dependency for anything else in oe-core or meta-openembedded.

Quoted 2 lines

- Trevor

I admit I didn't really pay attention to how well maintained the packages are, just fixed their run-time dependencies. I have nothing against removing this. This is precisely why I split the series into individual patches - some may be skipped while others go upstream.

Bart

#4 lore raw

PATCH 2/20 python3-attrs: unify RDEPENDS

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

The nativesdk and target RDEPENDS are the same and there's nothing that prohibits this package from build built for the native sysroot either. Use the global RDEPENDS instead of per-class assignments.

While at it: order the dependencies alphabetically.

---
 meta/recipes-devtools/python/python3-attrs_23.1.0.bb | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
meta/recipes-devtools/python/python3-attrs_23.1.0.bb +1 -5
index c8e2e514a4..207636d62e 100644
@@ -12,13 +12,9 @@ DEPENDS += " \
     ${PYTHON_PN}-hatch-fancy-pypi-readme-native \
 "
 
-RDEPENDS:${PN}:class-target += " \
-    ${PYTHON_PN}-crypt \
+RDEPENDS:${PN}+= " \
     ${PYTHON_PN}-ctypes \
-"
-RDEPENDS:${PN}:class-nativesdk += " \
     ${PYTHON_PN}-crypt \
-    ${PYTHON_PN}-ctypes \
 "
 
 BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#5 lore raw

PATCH 3/20 python3-attrs: don't use PYTHON_PN

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

We no longer support python2. Drop PYTHON_PN and use python3 explicitly.

---
 meta/recipes-devtools/python/python3-attrs_23.1.0.bb | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
meta/recipes-devtools/python/python3-attrs_23.1.0.bb +4 -4
index 207636d62e..ab3a08073d 100644
@@ -8,13 +8,13 @@ SRC_URI[sha256sum] = "6279836d581513a26f1bf235f9acd333bc9115683f14f7e8fae46c98fc
 inherit pypi python_hatchling
 
 DEPENDS += " \
-    ${PYTHON_PN}-hatch-vcs-native \
-    ${PYTHON_PN}-hatch-fancy-pypi-readme-native \
+    python3-hatch-vcs-native \
+    python3-hatch-fancy-pypi-readme-native \
 "
 
 RDEPENDS:${PN}+= " \
-    ${PYTHON_PN}-ctypes \
-    ${PYTHON_PN}-crypt \
+    python3-ctypes \
+    python3-crypt \
 "
 
 BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#6 lore raw

PATCH 4/20 python3-attrs: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-attrs_23.1.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/python/python3-attrs_23.1.0.bb +1 -0
index ab3a08073d..314053de08 100644
@@ -13,6 +13,7 @@ DEPENDS += " \
 "
 
 RDEPENDS:${PN}+= " \
+    python3-compression \
     python3-ctypes \
     python3-crypt \
 "
Signature · 1 line
2.39.2
#7 lore raw

PATCH 5/20 python3-certifi: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-certifi_2022.12.7.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/python/python3-certifi_2022.12.7.bb +2 -0
index dca3d26811..b2e3ec125a 100644
@@ -12,3 +12,5 @@ SRC_URI[sha256sum] = "35824b4c3a97115964b408844d64aa14db1cc518f6562e8d7261699d13
 inherit pypi setuptools3
 
 BBCLASSEXTEND = "native nativesdk"
+
+RDEPENDS:${PN} += "python3-io"
Signature · 1 line
2.39.2
#8 lore raw

PATCH 6/20 python3-libfdt: new package

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add a recipe for python bindings for libfdt (spun out of upstream dtc) fetched from pypi.

---
 .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb
meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb new file +20 -0
new file mode 100644
index 0000000000..1ff3decbb9
@@ -0,0 +1,20 @@
+SUMMARY = "Python bindings for libfdt."
+LICENSE = "GPL-2.0-only | BSD-2-Clause"
+LIC_FILES_CHKSUM = " \
+    file://BSD-2-Clause;md5=5d6306d1b08f8df623178dfd81880927 \
+    file://GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+"
+
+inherit pypi setuptools3
+
+PYPI_PACKAGE = "pylibfdt"
+
+SRC_URI[sha256sum] = "2d048f9f8ce9a0527d497f423dea1f1135f9811c05b009cc5d5753771c1f9ba1"
+
+BBCLASSEXTEND = "native nativesdk"
+
+DEPENDS += " \
+    python3-pip-native \
+    python3-setuptools-scm-native \
+    swig-native \
+"
Signature · 1 line
2.39.2
#9 lore raw

Re: [OE-core][PATCH 06/20] python3-libfdt: new package

Quoted 27 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add a recipe for python bindings for libfdt (spun out of upstream dtc) fetched from pypi.

---
  .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++++++++++++
  1 file changed, 20 insertions(+)

create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb

meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb new file +10 -0
new file mode 100644
index 0000000000..1ff3decbb9
@@ -0,0 +1,20 @@
+SUMMARY = "Python bindings for libfdt."
+LICENSE = "GPL-2.0-only | BSD-2-Clause"
+LIC_FILES_CHKSUM = " \
+    file://BSD-2-Clause;md5=5d6306d1b08f8df623178dfd81880927 \
+    file://GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+"
+
+inherit pypi setuptools3
+
+PYPI_PACKAGE = "pylibfdt"

meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb line 20

Quoted 10 lines
meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb new file +10 -0
+
+SRC_URI[sha256sum] = "2d048f9f8ce9a0527d497f423dea1f1135f9811c05b009cc5d5753771c1f9ba1"
+
+BBCLASSEXTEND = "native nativesdk"
+
+DEPENDS += " \
+    python3-pip-native \
+    python3-setuptools-scm-native \
+    swig-native \
+"

Can you also add this recipe to the meta/conf/distro/include/maintainers.inc file?

Quoted 9 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181455): https://lists.openembedded.org/g/openembedded-core/message/181455
Mute This Topic: https://lists.openembedded.org/mt/98944002/7611679
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#10 lore raw

Re: [OE-core][PATCH 06/20] python3-libfdt: new package

Quoted 33 lines · Trevor Gamblin
Quoted 31 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add a recipe for python bindings for libfdt (spun out of upstream dtc) fetched from pypi.

---
  .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++++++++++++
  1 file changed, 20 insertions(+)

create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb

meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb new file +14 -0
new file mode 100644
index 0000000000..1ff3decbb9
@@ -0,0 +1,20 @@
+SUMMARY = "Python bindings for libfdt."
+LICENSE = "GPL-2.0-only | BSD-2-Clause"
+LIC_FILES_CHKSUM = " \
+    file://BSD-2-Clause;md5=5d6306d1b08f8df623178dfd81880927 \
+    file://GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+"
+
+inherit pypi setuptools3
+
+PYPI_PACKAGE = "pylibfdt"
+
+SRC_URI[sha256sum] = "2d048f9f8ce9a0527d497f423dea1f1135f9811c05b009cc5d5753771c1f9ba1"
+
+BBCLASSEXTEND = "native nativesdk"
Quoted 8 lines

meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb line 20

Quoted 6 lines
meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb new file +6 -0
+
+DEPENDS += " \
+    python3-pip-native \
+    python3-setuptools-scm-native \
+    swig-native \
+"

Can you also add this recipe to the meta/conf/distro/include/maintainers.inc file?

As stated, this fails on the autobuilders:

The following recipes do not have a maintainer assigned to them. Please add an entry to meta/conf/distro/include/maintainers.inc file. python3-libfdt (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb)

I also got this:

stdio: WARNING: python3-libfdt-1.7.0.post1-r0 do_fetch: QA Issue: Recipe python3-libfdt in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb does not contain a HOMEPAGE. Please add an entry. [missing-metadata]
stdio: WARNING: python3-libfdt-1.7.0.post1-r0 do_fetch: QA Issue: Recipe python3-libfdt in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb does not have an assigned maintainer. Please add an entry into meta/conf/distro/include/maintainers.inc. [missing-metadata]
stdio: WARNING: python3-libfdt-native-1.7.0.post1-r0 do_fetch: QA Issue: Recipe python3-libfdt in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb does not contain a HOMEPAGE. Please add an entry. [missing-metadata]
stdio: WARNING: nativesdk-python3-libfdt-1.7.0.post1-r0 do_fetch: QA Issue: Recipe python3-libfdt in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb does not contain a HOMEPAGE. Please add an entry. [missing-metadata]

Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#11 lore raw

Re: [OE-core][PATCH 06/20] python3-libfdt: new package

In reply to Bartosz Golaszewski

I'm not sure if you saw the old threads on this, but we went through this binding in detail in february:

See the series from TrevorW:

[PATCH v3 1/4] dtc: version bump and add python wrapper

The pypi releases of libfdt bindings are not kept in sync with the dtc releases and rarely update. Not that they change a lot, but that is still the case.

The suggestion for the bindings is the same, they'd be better of built from dtc, versus using pypi.

There may be some issues remaining with the build, as Trevor hasn't updated his series in a while now.

Bruce

Quoted 39 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add a recipe for python bindings for libfdt (spun out of upstream dtc) fetched from pypi.

---
 .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb
meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb new file +20 -1
new file mode 100644
index 0000000000..1ff3decbb9
@@ -0,0 +1,20 @@
+SUMMARY = "Python bindings for libfdt."
+LICENSE = "GPL-2.0-only | BSD-2-Clause"
+LIC_FILES_CHKSUM = " \
+    file://BSD-2-Clause;md5=5d6306d1b08f8df623178dfd81880927 \
+    file://GPL;md5=b234ee4d69f5fce4486a80fdaf4a4263 \
+"
+
+inherit pypi setuptools3
+
+PYPI_PACKAGE = "pylibfdt"
+
+SRC_URI[sha256sum] = "2d048f9f8ce9a0527d497f423dea1f1135f9811c05b009cc5d5753771c1f9ba1"
+
+BBCLASSEXTEND = "native nativesdk"
+
+DEPENDS += " \
+    python3-pip-native \
+    python3-setuptools-scm-native \
+    swig-native \
+"
--
Quoted 2 lines

2.39.2

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#12 lore raw

Re: [OE-core][PATCH 06/20] python3-libfdt: new package

Quoted 15 lines · Bruce Ashfield via lists.openembedded.org

I'm not sure if you saw the old threads on this, but we went through this binding in detail in february:

See the series from TrevorW:

[PATCH v3 1/4] dtc: version bump and add python wrapper

The pypi releases of libfdt bindings are not kept in sync with the dtc releases and rarely update. Not that they change a lot, but that is still the case.

Quoted 5 lines

The suggestion for the bindings is the same, they'd be better of built from dtc, versus using pypi.

There may be some issues remaining with the build, as Trevor hasn't updated his series in a while now.

Agreed - that series needs to be resurrected as it’s the canonical source of the bindings.

Ross

#13 lore raw

Re: [OE-core][PATCH 06/20] python3-libfdt: new package

Quoted 24 lines · Ross Burton
Quoted 16 lines · Bruce Ashfield via lists.openembedded.org

I'm not sure if you saw the old threads on this, but we went through this binding in detail in february:

See the series from TrevorW:

[PATCH v3 1/4] dtc: version bump and add python wrapper

The pypi releases of libfdt bindings are not kept in sync with the dtc releases and rarely update. Not that they change a lot, but that is still the case.

Quoted 5 lines

The suggestion for the bindings is the same, they'd be better of built from dtc, versus using pypi.

There may be some issues remaining with the build, as Trevor hasn't updated his series in a while now.

Agreed - that series needs to be resurrected as it’s the canonical source of the bindings.

Quoted 9 lines

Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181527): https://lists.openembedded.org/g/openembedded-core/message/181527
Mute This Topic: https://lists.openembedded.org/mt/98944002/3618139
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

In this case I propose to just drop patches 06/20 and 07/20 from this series for now when applying.

Bart

#14 lore raw

PATCH 7/20 python3-dtschema: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-dtschema_2023.4.bb | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python3-dtschema_2023.4.bb +6 -1
index f87563facd..e5a18b0649 100644
@@ -10,6 +10,11 @@ PYPI_PACKAGE = "dtschema"
 SRC_URI[sha256sum] = "6daefb8f54403b4d82961b3346571200571747ab01950fd36c1f69950fa7a8cf"
 
 DEPENDS += "python3-setuptools-scm-native"
-RDEPENDS:${PN} += "python3-ruamel-yaml python3-jsonschema python3-rfc3987"
+RDEPENDS:${PN} += " \
+    python3-jsonschema \
+    python3-libfdt \
+    python3-rfc3987 \
+    python3-ruamel-yaml \
+"
 
 BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.39.2
#15 lore raw

PATCH 8/20 python3-hypothesis: fix run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

The main hypothesis module depends on pytest already so move it to global RDEPENDS from ptest-specific ones.

---
 meta/recipes-devtools/python/python3-hypothesis_6.71.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-hypothesis_6.71.0.bb +1 -1
index 8ec885fabf..15f4090c6d 100644
@@ -20,13 +20,13 @@ RDEPENDS:${PN} += " \
     python3-compression \
     python3-core \
     python3-json \
+    python3-pytest \
     python3-sortedcontainers \
     python3-statistics \
     python3-unittest \
     "
 
 RDEPENDS:${PN}-ptest += " \
-    ${PYTHON_PN}-pytest \
     ${PYTHON_PN}-unittest-automake-output \
     "
 
Signature · 1 line
2.39.2
#16 lore raw

PATCH 9/20 python3-tomli: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-tomli_2.0.1.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-devtools/python/python3-tomli_2.0.1.bb +5 -0
index 6118a6a9c3..9401ed897f 100644
@@ -11,3 +11,8 @@ inherit pypi python_flit_core
 SRC_URI[sha256sum] = "de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"
 
 BBCLASSEXTEND = "native nativesdk"
+
+RDEPENDS:${PN} += " \
+    python3-datetime \
+    python3-stringold \
+"
Signature · 1 line
2.39.2
#17 lore raw

PATCH 10/20 python3-pathspec: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-pathspec_0.11.1.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/python/python3-pathspec_0.11.1.bb +2 -0
index f49bf080e9..79f03da984 100644
@@ -9,3 +9,5 @@ SRC_URI[sha256sum] = "2798de800fa92780e33acca925945e9a19a133b715067cf165b8866c15
 inherit pypi setuptools3
 
 BBCLASSEXTEND = "native nativesdk"
+
+RDEPENDS:${PN} += "python3-profile"
Signature · 1 line
2.39.2
#18 lore raw

PATCH 11/20 python3-installer: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-installer_0.7.0.bb | 6 ++++++
 1 file changed, 6 insertions(+)
meta/recipes-devtools/python/python3-installer_0.7.0.bb +6 -0
index d7f1e79a5c..9429705b1f 100644
@@ -15,6 +15,12 @@ inherit pypi python_flit_core
 # Bootstrap the native build
 DEPENDS:remove:class-native = "python3-build-native python3-installer-native"
 
+RDEPENDS:${PN} += " \
+    python3-compile \
+    python3-compression \
+    python3-netclient \
+"
+
 INSTALL_WHEEL_COMPILE_BYTECODE:class-native = "--no-compile-bytecode"
 
 do_compile:class-native () {
Signature · 1 line
2.39.2
#19 lore raw

PATCH 12/20 python3-sphinx-rtd-theme: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

Remove DEPENDS as this package doesn't really have any.

---
 .../python/python3-sphinx-rtd-theme_1.2.0.bb                 | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python3-sphinx-rtd-theme_1.2.0.bb +4 -1
index e8a422b2e7..d9cd18e0ed 100644
@@ -6,7 +6,10 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=a1db7d4ef426c2935227264e1d4ae8f9 \
                     file://OFL-License.txt;md5=4534c22e0147eadb6828bd9fe86d4868 \
                     file://Apache-License-2.0.txt;md5=8a75796f0ef19c3f601d69857f5a9a5b"
 
-DEPENDS = "python3-sphinx"
+RDEPENDS:${PN} += " \
+    python3-compile \
+    python3-sphinx \
+"
 
 PYPI_PACKAGE = "sphinx_rtd_theme"
 
Signature · 1 line
2.39.2
#20 lore raw

PATCH 13/20 python3-setuptools-rust: fix RDEPENDS and allow target build

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Cargo and rustc can now be built for the target architecture. There's no reason to limit the setuptools rust extensions to native build only so make the RDEPENDS global. Also: add the missing ones.

---
 .../python/python3-setuptools-rust_1.5.2.bb     | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
meta/recipes-devtools/python/python3-setuptools-rust_1.5.2.bb +10 -7
index 502967fd20..01e29cc6d8 100644
@@ -18,13 +18,16 @@ inherit cargo pypi python_setuptools_build_meta
 
 DEPENDS += "python3-setuptools-scm-native python3-wheel-native"
 
-RDEPENDS:${PN}:class-native += " \
-    python3-semantic-version-native \
-    python3-setuptools-native \
-    python3-setuptools-scm-native \
-    python3-toml-native \
-    python3-typing-extensions-native \
-    python3-wheel-native \
+RDEPENDS:${PN} += " \
+    python3-distutils \
+    python3-json \
+    python3-semantic-version \
+    python3-setuptools \
+    python3-setuptools-scm \
+    python3-shell \
+    python3-toml \
+    python3-typing-extensions \
+    python3-wheel \
 "
 
 BBCLASSEXTEND = "native"
Signature · 1 line
2.39.2
#21 lore raw

PATCH 14/20 python3-pyproject-hooks: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 .../recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb +4 -1
index e3893d51cb..c3f1fb75ed 100644
@@ -14,7 +14,10 @@ BBCLASSEXTEND = "native nativesdk"
 # Bootstrap the native build
 DEPENDS:remove:class-native = "python3-build-native"
 
-RDEPENDS:${PN} += "python3-json"
+RDEPENDS:${PN} += " \
+    python3-io \
+    python3-json \
+"
 
 do_compile:class-native () {
     python_flit_core_do_manual_build
Signature · 1 line
2.39.2
#22 lore raw

PATCH 15/20 python3-pycryptodome: don't use PYTHON_PN

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

We no longer support python2. Drop PYTHON_PN and use python3 explicitly.

---
 meta/recipes-devtools/python/python-pycryptodome.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/recipes-devtools/python/python-pycryptodome.inc +3 -3
index 29fe80d224..86d7f261af 100644
@@ -10,12 +10,12 @@ inherit pypi
 PYPI_PACKAGE_EXT = "tar.gz"
 
 RDEPENDS:${PN} += " \
-    ${PYTHON_PN}-io \
-    ${PYTHON_PN}-math \
+    python3-io \
+    python3-math \
 "
 
 RDEPENDS:${PN}-tests += " \
-    ${PYTHON_PN}-unittest \
+    python3-unittest \
 "
 
 PACKAGES =+ "${PN}-tests"
Signature · 1 line
2.39.2
#23 lore raw

PATCH 16/20 python3-pycryptodome: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python-pycryptodome.inc | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/python/python-pycryptodome.inc +2 -0
index 86d7f261af..8d9b6d911e 100644
@@ -10,6 +10,8 @@ inherit pypi
 PYPI_PACKAGE_EXT = "tar.gz"
 
 RDEPENDS:${PN} += " \
+    python3-cffi \
+    python3-ctypes \
     python3-io \
     python3-math \
 "
Signature · 1 line
2.39.2
#24 lore raw

PATCH 17/20 python3-pygobject: add missing run-time dependencies

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Add missing RDEPENDS for this package.

---
 meta/recipes-devtools/python/python3-pygobject_3.44.1.bb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-devtools/python/python3-pygobject_3.44.1.bb +4 -1
index 8bfff43560..a5eecd33aa 100644
@@ -24,7 +24,10 @@ S = "${WORKDIR}/${SRCNAME}-${PV}"
 
 PACKAGECONFIG ??= "${@bb.utils.contains_any('DISTRO_FEATURES', [ 'directfb', 'wayland', 'x11' ], 'cairo', '', d)}"
 
-RDEPENDS:${PN} += "python3-pkgutil"
+RDEPENDS:${PN} += " \
+    python3-io \
+    python3-pkgutil \
+"
 
 # python3-pycairo is checked on configuration -> DEPENDS
 # we don't link against python3-pycairo -> RDEPENDS
Signature · 1 line
2.39.2
#25 lore raw

PATCH 18/20 python3-manifest: cgitb: new package

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Extend the manifest with definitions for the cgitb package from the Python standard library. Currently this package cannot be imported at run-time because python3-misc doesn't pull the required dependencies.

---
 .../python/python3/python3-manifest.json        | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
meta/recipes-devtools/python/python3/python3-manifest.json +17 -0
index f278b18775..f60c83d76b 100644
@@ -152,6 +152,23 @@
             "${libdir}/python${PYTHON_MAJMIN}/__pycache__/wave.*.pyc"
         ]
     },
+    "cgitb": {
+        "summary": "Special exception handler for Python scripts",
+        "rdepends": [
+            "core",
+            "crypt",
+            "html",
+            "io",
+            "math",
+            "pydoc"
+        ],
+        "files": [
+            "${libdir}/python${PYTHON_MAJMIN}/cgitb.py"
+        ],
+        "cached": [
+            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/cgitb.*.pyc"
+        ]
+    },
     "codecs": {
         "summary": "Python codec",
         "rdepends": [
Signature · 1 line
2.39.2
#26 lore raw

PATCH 19/20 python3-manifest: zipapp: new package

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Extend the manifest with definitions for the zipapp package from the Python standard library. Currently this package cannot be imported at run-time because python3-misc doesn't pull the required dependencies.

---
 .../python/python3/python3-manifest.json            | 13 +++++++++++++
 1 file changed, 13 insertions(+)
meta/recipes-devtools/python/python3/python3-manifest.json +13 -0
index f60c83d76b..d368b162bd 100644
@@ -1321,6 +1321,19 @@
         ],
         "cached": []
     },
+    "zipapp": {
+        "summary": "Tools to manage the creation of zip files containing Python code",
+        "rdepends": [
+            "compression",
+            "core"
+        ],
+        "files": [
+            "${libdir}/python${PYTHON_MAJMIN}/zipapp.py"
+        ],
+        "cached": [
+            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/zipapp.*.pyc"
+        ]
+    },
     "zoneinfo": {
         "summary": "IANA time zone support",
         "rdepends": [
Signature · 1 line
2.39.2
#27 lore raw

PATCH 20/20 python3-manifest: turtle: new package

In reply to Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

Extend the manifest with definitions for the turtle package from the Python standard library. Currently this package cannot be imported at run-time because python3-misc doesn't pull the required dependencies.

---
 .../python/python3/python3-manifest.json             | 12 ++++++++++++
 1 file changed, 12 insertions(+)
meta/recipes-devtools/python/python3/python3-manifest.json +12 -0
index d368b162bd..766f083ae4 100644
@@ -1236,6 +1236,18 @@
             "${libdir}/python${PYTHON_MAJMIN}/tomllib/_parser.*.pyc"
         ]
     },
+    "turtle": {
+        "summary": "Turtle graphics is a popular way for introducing programming to kids.",
+        "rdepends": [
+            "tkinter"
+        ],
+        "files": [
+            "${libdir}/python${PYTHON_MAJMIN}/turtle.py"
+        ],
+        "cached": [
+            "${libdir}/python${PYTHON_MAJMIN}/__pycache__/turtle.*.pyc"
+        ]
+     },
     "unittest": {
         "summary": "Python unit testing framework",
         "rdepends": [
Signature · 1 line
2.39.2
#28 lore raw
In reply to Bartosz Golaszewski

Thanks for this work. Perhaps we could catch this easier if there are runtime tests for those items, maybe in the shape of ptest.

Sometimes recipes have no longer needed dependencies too, and trimming those down is harder, but also appreciated.

Alex

Quoted 70 lines · Bartosz Golaszewski

From: Bartosz Golaszewski <[email protected]>

I noticed that every now and then I import some python package into an image only to find out the recipe didn't pull in all required run-time dependencies. I would fix it whenever I noticed it but figured that I could right a script that would go over all python packages, build a minimal image and see that all its modules can be imported.

The patches in this series are the result of that. Most simply add missing RDEPENDS, some are additional tweaks to recipes and the last three add new standard library packages to the manifest. These are the ones for which simply adding python3-misc would still be insufficient in terms of run-time dependencies.

I plan to do the same for meta-python but this will take more time and I need to think of some automated way of updating the recipes first.

Bartosz Golaszewski (20):
python3-async: add missing run-time dependencies
python3-attrs: unify RDEPENDS
python3-attrs: don't use PYTHON_PN
python3-attrs: add missing run-time dependencies
python3-certifi: add missing run-time dependencies
python3-libfdt: new package
python3-dtschema: add missing run-time dependencies
python3-hypothesis: fix run-time dependencies
python3-tomli: add missing run-time dependencies
python3-pathspec: add missing run-time dependencies
python3-installer: add missing run-time dependencies
python3-sphinx-rtd-theme: add missing run-time dependencies
python3-setuptools-rust: fix RDEPENDS and allow target build
python3-pyproject-hooks: add missing run-time dependencies
python3-pycryptodome: don't use PYTHON_PN
python3-pycryptodome: add missing run-time dependencies
python3-pygobject: add missing run-time dependencies
python3-manifest: cgitb: new package
python3-manifest: zipapp: new package
python3-manifest: turtle: new package

 meta/recipes-devtools/python/python-async.inc |  5 ++-
 .../python/python-pycryptodome.inc            |  8 ++--
 .../python/python3-attrs_23.1.0.bb            | 15 +++----
 .../python/python3-certifi_2022.12.7.bb       |  2 +
 .../python/python3-dtschema_2023.4.bb         |  7 +++-
 .../python/python3-hypothesis_6.71.0.bb       |  2 +-
 .../python/python3-installer_0.7.0.bb         |  6 +++
 .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++
 .../python/python3-pathspec_0.11.1.bb         |  2 +
 .../python/python3-pygobject_3.44.1.bb        |  5 ++-
 .../python/python3-pyproject-hooks_1.0.0.bb   |  5 ++-
 .../python/python3-setuptools-rust_1.5.2.bb   | 17 ++++----
 .../python/python3-sphinx-rtd-theme_1.2.0.bb  |  5 ++-
 .../python/python3-tomli_2.0.1.bb             |  5 +++
 .../python/python3/python3-manifest.json      | 42 +++++++++++++++++++
 15 files changed, 121 insertions(+), 25 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3-libfdt_1.7.0.post1.bb

--
2.39.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181449): https://lists.openembedded.org/g/openembedded-core/message/181449
Mute This Topic: https://lists.openembedded.org/mt/98943996/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#29 lore raw

Thank you, Bartosz. I didn't review in great depth, but others have done so. This series looks like a
step in the right direction.

Quoted 7 lines · Alexander Kanavin

Thanks for this work. Perhaps we could catch this easier if there are runtime tests for those items, maybe in the shape of ptest.

Sometimes recipes have no longer needed dependencies too, and trimming those down is harder, but also appreciated.

One challenge has been the additional dependencies that come in for

ptests. Another is that
many PyPi packages do not include their /tests/ directory, meaning we would need to switch to
the git fetcher in many cases. My intent has always been "full" ptest coverage, when practical.

Alex

Quoted 79 lines · Bartosz Golaszewski
Quoted 49 lines

From: Bartosz Golaszewski <[email protected]>

I noticed that every now and then I import some python package into an image only to find out the recipe didn't pull in all required run-time dependencies. I would fix it whenever I noticed it but figured that I could right a script that would go over all python packages, build a minimal image and see that all its modules can be imported.

The patches in this series are the result of that. Most simply add missing RDEPENDS, some are additional tweaks to recipes and the last three add new standard library packages to the manifest. These are the ones for which simply adding python3-misc would still be insufficient in terms of run-time dependencies.

I plan to do the same for meta-python but this will take more time and I need to think of some automated way of updating the recipes first.

Bartosz Golaszewski (20):
python3-async: add missing run-time dependencies
python3-attrs: unify RDEPENDS
python3-attrs: don't use PYTHON_PN
python3-attrs: add missing run-time dependencies
python3-certifi: add missing run-time dependencies
python3-libfdt: new package
python3-dtschema: add missing run-time dependencies
python3-hypothesis: fix run-time dependencies
python3-tomli: add missing run-time dependencies
python3-pathspec: add missing run-time dependencies
python3-installer: add missing run-time dependencies
python3-sphinx-rtd-theme: add missing run-time dependencies
python3-setuptools-rust: fix RDEPENDS and allow target build
python3-pyproject-hooks: add missing run-time dependencies
python3-pycryptodome: don't use PYTHON_PN
python3-pycryptodome: add missing run-time dependencies
python3-pygobject: add missing run-time dependencies
python3-manifest: cgitb: new package
python3-manifest: zipapp: new package
python3-manifest: turtle: new package

 meta/recipes-devtools/python/python-async.inc |  5 ++-
 .../python/python-pycryptodome.inc            |  8 ++--
 .../python/python3-attrs_23.1.0.bb            | 15 +++----
 .../python/python3-certifi_2022.12.7.bb       |  2 +
 .../python/python3-dtschema_2023.4.bb         |  7 +++-
 .../python/python3-hypothesis_6.71.0.bb       |  2 +-
 .../python/python3-installer_0.7.0.bb         |  6 +++
 .../python/python3-libfdt_1.7.0.post1.bb      | 20 +++++++++
 .../python/python3-pathspec_0.11.1.bb         |  2 +
Quoted 8 lines
 .../python/python3-pygobject_3.44.1.bb        |  5 ++-
 .../python/python3-pyproject-hooks_1.0.0.bb   |  5 ++-
 .../python/python3-setuptools-rust_1.5.2.bb   | 17 ++++----
 .../python/python3-sphinx-rtd-theme_1.2.0.bb  |  5 ++-
 .../python/python3-tomli_2.0.1.bb             |  5 +++
 .../python/python3/python3-manifest.json      | 42 +++++++++++++++++++
 15 files changed, 121 insertions(+), 25 deletions(-)
 create mode 100644 meta/recipes-devtools/python/

python3-libfdt_1.7.0.post1.bb

Quoted 7 lines

--
2.39.2

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#181479):
https://lists.openembedded.org/g/openembedded-core/message/181479
Mute This Topic: https://lists.openembedded.org/mt/98943996/924729
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-