package/ninja: invoke python3 explicitly to configure the package
PATCH v2 1/2 package/ninja: invoke python3 explicitly to configure the package
ninja depends on python3 specifically, but the configure.py file simply uses "env python". Where no python is selected for the target you simply won't get a python symlink in $(HOST_DIR)/usr/bin, so the configure.py script fails to run since it can't find "python".
Notice that in order to reproduce the issue, you must not have python2 installed on your host machine.
--- package/ninja/ninja.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
package/ninja/ninja.mk +1 -1
@@ -17,7 +17,7 @@ NINJA_LICENSE_FILES = COPYING HOST_NINJA_DEPENDENCIES = host-python3 define HOST_NINJA_BUILD_CMDS - (cd $(@D); ./configure.py --bootstrap) + (cd $(@D); $(HOST_DIR)/bin/python3 configure.py --bootstrap) endef define HOST_NINJA_INSTALL_CMDS
Signature · 1 line
2.17.1
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
qemu requires python in its configre script. Yet host-python was not listed as one of the package's dependencies. If no other package requested host-python, then configuring this package will fail since it won't find any executable named python in the host dir.
In order to reproduce this issue you must not have python2 installed on your host machine.
--- package/qemu/qemu.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
package/qemu/qemu.mk +2 -2
@@ -21,7 +21,7 @@ QEMU_LICENSE_FILES = COPYING COPYING.LIB #------------------------------------------------------------- # Target-qemu -QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman +QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman host-python # Need the LIBS variable because librt and libm are # not automatically pulled. :-(
@@ -182,7 +182,7 @@ $(eval $(generic-package)) #------------------------------------------------------------- # Host-qemu -HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman +HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman host-python # BR ARCH qemu # ------- ----
Signature · 1 line
2.17.1
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
Quoted 9 lines · On Mon, 11 Nov 2019 23:31:46 +0200 Avi Shukron <[email protected]>…
qemu requires python in its configre script. Yet host-python was not listed as one of the package's dependencies. If no other package requested host-python, then configuring this package will fail since it won't find any executable named python in the host dir.
In order to reproduce this issue you must not have python2 installed on your host machine.
Signed-off-by Avi Shukron <[email protected]>
qemu needs python 2 *or* python 3, so we can do something like this:
QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
This way, if you have Python 3.x enabled for the target (and therefore you are already building host-python3), it will use host-python3.
Could you test this in a configuration that has BR2_PACKAGE_QEMU=y and BR2_PACKAGE_PYTHON3=y, and check it works properly ?
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
Quoted 5 lines · Thomas Petazzoni
qemu needs python 2 *or* python 3, so we can do something like this:
QEMU_DEPENDENCIES= host-pkgconf libglib2 zlib pixman $(if
$(BR2_PACKAGE_PYTHON3),host-python3,host-python)
Quoted 6 lines
This way, if you have Python 3.x enabled for the target (and therefore you are already building host-python3), it will use host-python3.
Could you test this in a configuration that has BR2_PACKAGE_QEMU=y and BR2_PACKAGE_PYTHON3=y, and check it works properly ?
Your suggestion works. If python3 is selected - host-python3 is used. BUT - host-qemu also depends on host-libglib2 which depends on host-meson which depends on python3,
so even if you **don't** select python3 you end up with host-python3 because of meson, and also
host-python because now host-qemu directly depends on it. Attached a dependency graph depicting
the scenario.
So - since host-python3 will be used anyway - I suggest we just depend on it exclusively.
What do you think?
BTW - I'm pretty sure that there aren't really so many packages that truly depends on **host**-python2 (except maybe python2 itself)
So in general I think it could be a good idea to just replace **all** host-python with host-python3 (again - except python2 itself) and just see what happens.
It might just work.
Avi.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191113/a331bcf6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: host-qemu-graph-depends.png
Type: image/png
Size: 203002 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191113/a331bcf6/attachment-0001.png>
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
Hello,
Quoted 3 lines · On Wed, 13 Nov 2019 22:22:00 +0200 Avraham Shukron <[email protected]>…
Your suggestion works. If python3 is selected - host-python3 is used. BUT - host-qemu also depends on host-libglib2 which depends on host-meson which depends on python3,
True. But then, how did you get build issues when building qemu ? I mean, your v3 in fact doesn't do anything really: host-python3 was already built as one of the dependencies of qemu, through libglib2. So how did you get issues ?
Quoted 5 lines
BTW - I'm pretty sure that there aren't really so many packages that truly depends on **host**-python2 (except maybe python2 itself)
So in general I think it could be a good idea to just replace **all** host-python with host-python3 (again - except python2 itself) and just see what happens.
We discussed this at the latest Buildroot Developers meeting, you can see some notes in "Python 2 deprecation" at
https://bimestriel.framapad.org/p/buildroot-elce-2019-meeting. We welcome contributions in this area.
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
Quoted 4 lines · Thomas Petazzoni
True. But then, how did you get build issues when building qemu ? I mean, your v3 in fact doesn't do anything really: host-python3 was already built as one of the dependencies of qemu, through libglib2. So how did you get issues ?
Damn, you are right, host-python3 won't work.
I've managed to build it only because I had host-python(2) still installed in my host dir during my experiments.
A clean build still fails because there is no "python" executable in the host dir unless you build any python for
the target.
So we only left with your suggestion, that in the best case (python3 is selected for the target)
will only build host-python3.
Any other case will force both host-python and host-python3 to be built.
So I'll re-submit with your $(if..) solution soon.
Quoted 4 lines
We discussed this at the latest Buildroot Developers meeting, you can see some notes in "Python 2 deprecation" at
https://bimestriel.framapad.org/p/buildroot-elce-2019-meeting. We welcome contributions in this area.
Great, I'll try to put some work into that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191116/34ead835/attachment.html>
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
Hello,
Quoted 7 lines · On Sat, 16 Nov 2019 00:04:59 +0200 Avraham Shukron <[email protected]>…
Damn, you are right, host-python3 won't work.
I've managed to build it only because I had host-python(2) still installed in my host dir during my experiments.
A clean build still fails because there is no "python" executable in the host dir unless you build any python for
the target.
Quoted 6 lines
So we only left with your suggestion, that in the best case (python3 is selected for the target)
will only build host-python3.
Any other case will force both host-python and host-python3 to be built.So I'll re-submit with your $(if..) solution soon.
So, the qemu build system requires having an interpreter called "python" ?
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 2/2 package/qemu: add host-python as an explicit dependency
Quoted 2 lines · Thomas Petazzoni
So, the qemu build system requires having an interpreter called "python" ?
Yes.
But now I see that I can pass --python to the configure script with
the path to the interpreter, so I can simply pass $(HOST_DIR)/bin/python3
and
be done with it...
One less package depending on host-python(2).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191116/9e191f39/attachment.html>
PATCH v4 1/1 package/qemu: add host-python3 as an explicit dependency
qemu requires python in its configre script. Yet host-python was not listed as one of the package's dependencies. If no other package requested host-python, then configuring this package will fail since it won't find any executable named python in the host dir.
In order to reproduce this issue you must not have python2 installed on your host machine.
---
Changes v2->v3:
- Depend directly on host-python3, since it is already a transitive dependency of qemu
Changes v3->v4:
- Pass python3 explicitly to the configure script
--- package/qemu/qemu.mk | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
package/qemu/qemu.mk +4 -3
@@ -20,8 +20,7 @@ QEMU_LICENSE_FILES = COPYING COPYING.LIB #------------------------------------------------------------- # Target-qemu - -QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman +QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman host-python3 # Need the LIBS variable because librt and libm are # not automatically pulled. :-(
@@ -132,6 +131,7 @@ define QEMU_CONFIGURE_CMDS --prefix=/usr \ --cross-prefix=$(TARGET_CROSS) \ --audio-drv-list= \ + --python=$(HOST_DIR)/bin/python3 \ --enable-kvm \ --enable-attr \ --enable-vhost-net \
@@ -182,7 +182,7 @@ $(eval $(generic-package)) #------------------------------------------------------------- # Host-qemu -HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman +HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman host-python3 # BR ARCH qemu # ------- ----
@@ -299,6 +299,7 @@ define HOST_QEMU_CONFIGURE_CMDS --host-cc="$(HOSTCC)" \ --extra-cflags="$(HOST_CFLAGS)" \ --extra-ldflags="$(HOST_LDFLAGS)" \ + --python=$(HOST_DIR)/bin/python3 \ $(HOST_QEMU_OPTS) endef
Signature · 1 line
2.17.1
PATCH v3 1/1 package/qemu: add host-python as an explicit dependency
In reply to Thomas Petazzoniqemu requires python in its configre script. Yet host-python was not listed as one of the package's dependencies. If no other package requested host-python, then configuring this package will fail since it won't find any executable named python in the host dir.
In order to reproduce this issue you must not have python2 installed on your host machine.
---
Changes v2->v3:
- Depend directly on host-python3, since it is already a transitive dependency of qemu
--- package/qemu/qemu.mk | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
package/qemu/qemu.mk +2 -3
@@ -20,8 +20,7 @@ QEMU_LICENSE_FILES = COPYING COPYING.LIB #------------------------------------------------------------- # Target-qemu - -QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman +QEMU_DEPENDENCIES = host-pkgconf libglib2 zlib pixman host-python3 # Need the LIBS variable because librt and libm are # not automatically pulled. :-(
@@ -182,7 +181,7 @@ $(eval $(generic-package)) #------------------------------------------------------------- # Host-qemu -HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman +HOST_QEMU_DEPENDENCIES = host-pkgconf host-zlib host-libglib2 host-pixman host-python3 # BR ARCH qemu # ------- ----
Signature · 1 line
2.17.1
Thomas Petazzoni
AppliedPATCH v3 1/1 package/qemu: add host-python as an explicit dependency
Quoted 8 lines · On Fri, 15 Nov 2019 22:19:00 +0200 Avi Shukron <[email protected]>…
qemu requires python in its configre script. Yet host-python was not listed as one of the package's dependencies. If no other package requested host-python, then configuring this package will fail since it won't find any executable named python in the host dir.
In order to reproduce this issue you must not have python2 installed on your host machine.
Quoted 8 lines
Signed-off-by Avi Shukron <[email protected]>---
Changesv2->v3:
- Depend directly on host-python3, since it is already a transitive dependency of qemu--- package/qemu/qemu.mk | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
Applied to master, thanks.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Thomas Petazzoni
AppliedPATCH v2 1/2 package/ninja: invoke python3 explicitly to configure the package
In reply to Avi ShukronQuoted 5 lines · On Mon, 11 Nov 2019 23:31:45 +0200 Avi Shukron <[email protected]>…
ninja depends on python3 specifically, but the
configure.pyfile simply uses "env python". Where no python is selected for the target you simply won't get a python symlink in $(HOST_DIR)/usr/bin, so theconfigure.pyscript fails to run since it can't find "python".
Quoted 7 lines
Notice that in order to reproduce the issue, you must not have python2 installed on your host machine.
Signed-off-by Avi Shukron <[email protected]>--- package/ninja/ninja.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Applied to master, thanks.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH v2 1/2 package/ninja: invoke python3 explicitly to configure the package
In reply to Avi ShukronQuoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Avi" == Avi Shukron <[email protected]> writes:
ninja depends on python3 specifically, but the
configure.pyfile simply uses "env python". Where no python is selected for the target you simply won't get a python symlink in $(HOST_DIR)/usr/bin, so theconfigure.pyscript fails to run since it can't find "python".Notice that in order to reproduce the issue, you must not have python2 installed on your host machine.
Signed-off-by Avi Shukron <[email protected]>
Committed to 2019.08.x, thanks.
Signature · 1 line
Bye, Peter Korsgaard