python*-setuptools: add separate packages for pkg_resources module
PATCH 1/1 python*-setuptools: add separate packages for pkg_resources module
From: Luca Boccassi <[email protected]>
The pkg_resources Python module is useful by itself, for example for automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.
Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages, to avoid regressions for other packages that depend on them already.
--- meta/recipes-devtools/python/python-setuptools.inc | 4 ++++ 1 file changed, 4 insertions(+)
meta/recipes-devtools/python/python-setuptools.inc +4 -0
@@ -24,6 +24,7 @@ RDEPENDS_${PN} = "\ ${PYTHON_PN}-html \ ${PYTHON_PN}-netserver \ ${PYTHON_PN}-numbers \ + ${PYTHON_PN}-pkg-resources \ ${PYTHON_PN}-pkgutil \ ${PYTHON_PN}-plistlib \ ${PYTHON_PN}-shell \
@@ -37,3 +38,6 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources = "${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
Signature · 1 line
2.20.1
Quoted 41 lines · On Thu, 2019-05-16 at 22:09 +0100, [email protected] wrote:
From: Luca Boccassi <[email protected]>
The
pkg_resourcesPython module is useful by itself, for example for automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages, to avoid regressions for other packages that depend on them already.
Signed-off-by Luca Boccassi <[email protected]>--- meta/recipes-devtools/python/python-setuptools.inc | 4 ++++ 1 file changed, 4 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +4
-0@@ -24,6 +24,7 @@ RDEPENDS_${PN} = "\ ${PYTHON_PN}-html \ ${PYTHON_PN}-netserver \ ${PYTHON_PN}-numbers \ + ${PYTHON_PN}-pkg-resources \ ${PYTHON_PN}-pkgutil \ ${PYTHON_PN}-plistlib \ ${PYTHON_PN}-shell \@@ -37,3 +38,6 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources =
Quoted 1 line
"${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
This breaks native builds:
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/614
(and many other similar failures)
Cheers,
Richard
Quoted 55 lines · Richard Purdie
Quoted 47 lines · On Thu, 2019-05-16 at 22:09 +0100, [email protected] wrote:
From: Luca Boccassi <
[email protected]Quoted 1 line
The
pkg_resourcesPython module is useful by itself, for example for
automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages,
to avoid regressions for other packages that depend on them already.Signed-off-by: Luca Boccassi <
[email protected]Quoted 1 line
--- meta/recipes-devtools/python/python-setuptools.inc | 4 ++++ 1 file changed, 4 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +4
-0@@ -24,6 +24,7 @@ RDEPENDS_${PN} = "\ ${PYTHON_PN}-html \ ${PYTHON_PN}-netserver \ ${PYTHON_PN}-numbers \ + ${PYTHON_PN}-pkg-resources \ ${PYTHON_PN}-pkgutil \ ${PYTHON_PN}-plistlib \ ${PYTHON_PN}-shell \@@ -37,3 +38,6 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources =Quoted 1 line
"${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
This breaks native builds:
Quoted 8 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/614
(and many other similar failures)
Cheers,
Richard
Hello Richard,
Thanks for the heads-up, any suggestion as to why the above patch
wouldn't be enough to build both native and target binary packages for $python-pkg-resources?
Signature · 2 lines
Kind regards, Luca Boccassi
I think you need to reset RDEPENDS to an empty string for the native class. This is a long-standing problem, where bitbake is not able to fulfil RDEPENDS for native variants, if said RDEPENDS contains anything except recipe names.
Alex
Quoted 80 lines · Luca Boccassi
Quoted 55 lines · Richard Purdie
Quoted 47 lines · On Thu, 2019-05-16 at 22:09 +0100, [email protected] wrote:
From: Luca Boccassi <
[email protected]Quoted 1 line
The
pkg_resourcesPython module is useful by itself, for example for
automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages,
to avoid regressions for other packages that depend on them already.Signed-off-by: Luca Boccassi <
[email protected]Quoted 1 line
--- meta/recipes-devtools/python/python-setuptools.inc | 4 ++++ 1 file changed, 4 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +4
-0@@ -24,6 +24,7 @@ RDEPENDS_${PN} = "\ ${PYTHON_PN}-html \ ${PYTHON_PN}-netserver \ ${PYTHON_PN}-numbers \ + ${PYTHON_PN}-pkg-resources \ ${PYTHON_PN}-pkgutil \ ${PYTHON_PN}-plistlib \ ${PYTHON_PN}-shell \@@ -37,3 +38,6 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources =Quoted 1 line
"${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
This breaks native builds:
Quoted 8 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/614
(and many other similar failures)
Cheers,
Richard
Hello Richard,
Thanks for the heads-up, any suggestion as to why the above patch
wouldn't be enough to build both native and target binary packages for $python-pkg-resources?--
Kind regards,
Luca Boccassi--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
PATCH v2 1/1 python*-setuptools: add separate packages for pkg_resources module
In reply to [email protected]From: Luca Boccassi <[email protected]>
The pkg_resources Python module is useful by itself, for example for automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.
Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages, to avoid regressions for other packages that depend on them already.
---
v2: restrict new RDEPENDS to class-target. As advised by Alexander, bitbake
cannot resolve native rdeps that mention package names rather than recipe names.
meta/recipes-devtools/python/python-setuptools.inc | 6 ++++++ 1 file changed, 6 insertions(+)
meta/recipes-devtools/python/python-setuptools.inc +6 -0
@@ -37,3 +37,9 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources = "${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*" +# Due to a bitbake bug, the native class cannot have a dependency on something +# that is not a recipe name. +RDEPENDS_${PN}_append_class-target = " ${PYTHON_PN}-pkg-resources"
Signature · 1 line
2.20.1
Quoted 42 lines · On Tue, 2019-05-21 at 11:59 +0100, [email protected] wrote:
From: Luca Boccassi <[email protected]>
The
pkg_resourcesPython module is useful by itself, for example for automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages, to avoid regressions for other packages that depend on them already.
Signed-off-by Luca Boccassi <[email protected]>---
v2: restrict new RDEPENDS to class-target. As advised by Alexander, bitbake
cannot resolve native rdeps that mention package names rather
than
recipe names.meta/recipes-devtools/python/python-setuptools.inc | 6 ++++++ 1 file changed, 6 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +3
-0@@ -37,3 +37,9 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources ="${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
+# Due to a bitbake bug, the native class cannot have a dependency on something
meta/recipes-devtools/python/python-setuptools.inc do_install_prepend() { · line 45
Quoted 2 lines
meta/recipes-devtools/python/python-setuptools.inc +2
-0+# that is not a recipe name. +RDEPENDS_${PN}_append_class-target = " ${PYTHON_PN}-pkg-resources"
Its not a bitbake bug as such but the way OE-Core decided to implement native recipes.
I'm guessing but the best thing to do here is probably add:
RPROVIDES_append_class-native = " ${PYTHON_PN}-pkg-resources-native"
which is slightly cleaner than the target change above (I'm left wondering if nativesdk works in that case).
Cheers,
Richard
PATCH v4 1/1 python*-setuptools: add separate packages for pkg_resources module
In reply to [email protected]From: Luca Boccassi <[email protected]>
The pkg_resources Python module is useful by itself, for example for automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.
Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages, to avoid regressions for other packages that depend on them already.
---
v2: restrict new RDEPENDS to class-target. As advised by Alexander, bitbake
cannot resolve native rdeps that mention package names rather than recipe names.
v3: manually add RPROVIDES to the native class instead of restricting the
RDEPENDS to the target class as a better workaround. Also document why the package is being split.
v4: re-send to the correct thread, no changes.
meta/recipes-devtools/python/python-setuptools.inc | 11 +++++++++++ 1 file changed, 11 insertions(+)
meta/recipes-devtools/python/python-setuptools.inc +11 -0
@@ -37,3 +37,14 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +# The pkg-resources module can be used by itself, without the package downloader +# and easy_install. Ship it in a separate package so that it can be used by +# minimal distributions. +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources = "${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*" +# Due to the way OE-Core implemented native recipes, the native class cannot +# have a dependency on something that is not a recipe name. Work around that by +# manually setting RPROVIDES. +RDEPENDS_${PN}_append = " ${PYTHON_PN}-pkg-resources" +RPROVIDES_append_class-native = " ${PYTHON_PN}-pkg-resources-native"
Signature · 1 line
2.20.1
Quoted 39 lines · On Tue, May 21, 2019 at 5:36 AM <[email protected]> wrote:
From: Luca Boccassi <[email protected]>
The
pkg_resourcesPython module is useful by itself, for example for automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages, to avoid regressions for other packages that depend on them already.
Signed-off-by Luca Boccassi <[email protected]>---
v2: restrict new RDEPENDS to class-target. As advised by Alexander, bitbake
cannot resolve native rdeps that mention package names rather than recipe names.
v3: manually add RPROVIDES to the native class instead of restricting the
RDEPENDS to the target class as a better workaround. Also document why the package is being split.
v4: re-send to the correct thread, no changes.meta/recipes-devtools/python/python-setuptools.inc | 11 +++++++++++ 1 file changed, 11 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +1
-0@@ -37,3 +37,14 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" +
meta/recipes-devtools/python/python-setuptools.inc do_install_prepend() { · line 50
Quoted 10 lines
meta/recipes-devtools/python/python-setuptools.inc +10
-0+# The pkg-resources module can be used by itself, without the package downloader +# and easy_install. Ship it in a separate package so that it can be used by +# minimal distributions. +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources = "${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*" +# Due to the way OE-Core implemented native recipes, the native class cannot +# have a dependency on something that is not a recipe name. Work around that by +# manually setting RPROVIDES. +RDEPENDS_${PN}_append = " ${PYTHON_PN}-pkg-resources" +RPROVIDES_append_class-native = " ${PYTHON_PN}-pkg-resources-native"
do we need to handle nativesdk case ?
Quoted 7 lines
2.20.1
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 63 lines · Khem Raj
On Tue, May 21, 2019 at 5:36 AM <
[email protected]Quoted 61 lines
wrote:
From: Luca Boccassi <
[email protected]Quoted 1 line
The
pkg_resourcesPython module is useful by itself, for example for
automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages,
to avoid regressions for other packages that depend on them already.Signed-off-by: Luca Boccassi <
[email protected]Quoted 1 line
---
v2: restrict new RDEPENDS to class-target. As advised by Alexander, bitbake
cannot resolve native rdeps that mention package names rather
than
recipe names.
v3: manually add RPROVIDES to the native class instead of restricting the
RDEPENDS to the target class as a better workaround. Also
document why
the package is being split.
v4: re-send to the correct thread, no changes.meta/recipes-devtools/python/python-setuptools.inc | 11 +++++++++++ 1 file changed, 11 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +2
-0@@ -37,3 +37,14 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +# The pkg-resources module can be used by itself, without thepackage downloader
+# andeasy_install. Ship it in a separate package so that it can be used bymeta/recipes-devtools/python/python-setuptools.inc +3
-0+# minimal distributions. +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources ="${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
+# Due to the way OE-Core implemented native recipes, the native class cannot
Quoted 8 lines
Quoted 6 lines
+# have a dependency on something that is not a recipe name. Work around that by
meta/recipes-devtools/python/python-setuptools.inc +3
-0+# manually setting RPROVIDES. +RDEPENDS_${PN}_append = " ${PYTHON_PN}-pkg-resources" +RPROVIDES_append_class-native = " ${PYTHON_PN}-pkg-resources-native"
do we need to handle nativesdk case ?
Hi,
The parsing step of "bitbake core-image-minimal -c populate_sdk" works, while without the append_class-native workaround it fails immediately. Is this enough or is there something else I should run to check?
Thanks!
Signature · 2 lines
Kind regards, Luca Boccassi
Quoted 74 lines · Luca Boccassi
Quoted 63 lines · Khem Raj
On Tue, May 21, 2019 at 5:36 AM <
[email protected]Quoted 61 lines
wrote:
From: Luca Boccassi <
[email protected]Quoted 1 line
The
pkg_resourcesPython module is useful by itself, for example for
automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools and pkg-resources in this way.
The setuptools packages now depend on the new pkg-resources packages,
to avoid regressions for other packages that depend on them already.Signed-off-by: Luca Boccassi <
[email protected]Quoted 1 line
---
v2: restrict new RDEPENDS to class-target. As advised by Alexander, bitbake
cannot resolve native rdeps that mention package names rather
than
recipe names.
v3: manually add RPROVIDES to the native class instead of restricting the
RDEPENDS to the target class as a better workaround. Also
document why
the package is being split.
v4: re-send to the correct thread, no changes.meta/recipes-devtools/python/python-setuptools.inc | 11 +++++++++++ 1 file changed, 11 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +2
-0@@ -37,3 +37,14 @@ do_install_prepend() { } BBCLASSEXTEND = "native nativesdk" + +# The pkg-resources module can be used by itself, without thepackage downloader
+# andeasy_install. Ship it in a separate package so that it can be used bymeta/recipes-devtools/python/python-setuptools.inc +3
-0+# minimal distributions. +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources ="${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
+# Due to the way OE-Core implemented native recipes, the native class cannotQuoted 8 lines
Quoted 6 lines
+# have a dependency on something that is not a recipe name. Work around that by
meta/recipes-devtools/python/python-setuptools.inc +3
-0+# manually setting RPROVIDES. +RDEPENDS_${PN}_append = " ${PYTHON_PN}-pkg-resources" +RPROVIDES_append_class-native = " ${PYTHON_PN}-pkg-resources-native"
do we need to handle nativesdk case ?
Quoted 6 lines
Hi,
The parsing step of "bitbake core-image-minimal -c
populate_sdk" works, while without the append_class-native workaround it fails immediately. Is this enough or is there something else I should run to check?
yeah usually to test nativesdk we need to do it with generated SDK
Quoted 6 lines
Thanks!
--
Kind regards,
Luca Boccassi
Quoted 86 lines · Khem Raj
Quoted 84 lines · Luca Boccassi
Quoted 71 lines · Khem Raj
On Tue, May 21, 2019 at 5:36 AM <
[email protected]Quoted 69 lines
wrote:
From: Luca Boccassi <
[email protected]Quoted 1 line
The
pkg_resourcesPython module is useful by itself, for example
for
automatic loading of resources shipped in a Python package. Add separate packages for it, so that users can depend on them individually and avoid pulling in the entire setuptools, which include scripts to download other packages, which might not be desired on minimal images.Other distributions like Debian and Ubuntu already split setuptools
and pkg-resources in this way.The setuptools packages now depend on the new pkg-resources packages,
to avoid regressions for other packages that depend on them already.Signed-off-by: Luca Boccassi <
[email protected]Quoted 1 line
---
v2: restrict new RDEPENDS to class-target. As advised by Alexander,
bitbake
cannot resolve native rdeps that mention package names rather
than
recipe names.
v3: manually add RPROVIDES to the native class instead of restricting the
RDEPENDS to the target class as a better workaround. Also document why
the package is being split.
v4: re-send to the correct thread, no changes.meta/recipes-devtools/python/python-setuptools.inc | 11
+++++++++++1 file changed, 11 insertions(+)meta/recipes-devtools/python/python-setuptools.inc +0
-0@@ -37,3 +37,14 @@ do_install_prepend() { }BBCLASSEXTEND = "native nativesdk"
meta/recipes-devtools/python/python-setuptools.inc +2
-0+ +# The pkg-resources module can be used by itself, without thepackage downloader
+# andeasy_install. Ship it in a separate package so that it can
be used bymeta/recipes-devtools/python/python-setuptools.inc +3
-0+# minimal distributions. +PACKAGES =+ "${PYTHON_PN}-pkg-resources " +FILES_${PYTHON_PN}-pkg-resources ="${PYTHON_SITEPACKAGES_DIR}/pkg_resources/*"
+# Due to the way OE-Core implemented native recipes, the native
class cannot
+# have a dependency on something that is not a recipe name.Quoted 8 lines
Quoted 6 lines
Work
around that bymeta/recipes-devtools/python/python-setuptools.inc +3
-0+# manually setting RPROVIDES. +RDEPENDS_${PN}_append = " ${PYTHON_PN}-pkg-resources" +RPROVIDES_append_class-native = " ${PYTHON_PN}-pkg-resources-native"
do we need to handle nativesdk case ?
Hi,
Quoted 8 lines
Quoted 6 lines
The parsing step of "bitbake core-image-minimal -c
populate_sdk" works,
while without the append_class-native workaround it fails immediately.
Is this enough or is there something else I should run to check?yeah usually to test nativesdk we need to do it with generated SDK
I do have a nativesdk target in the distro at $work, but I realise it might be different enough. Is there a simple config change/command to generate it from Poky?
Signature · 2 lines
Kind regards, Luca Boccassi