CFFI backend
v2 cover CFFI backend
This patch series introduces CFFI backend for Python packages. CFFI can cross-compile C library wrappers and create *.py and *.so files for the target rootfs. In order to do so CFFI uses pycparser to parse C heasers/sources. The target variant of CFFI is working with already prepared *.py and *.so files and doesn't need pycparser any longer.
The libffi headers patch is needed, because only headers for target variant will be copied to the proper destination and host packages, that don't use pkg-config cannot find libffi headers at its default location.
Yegor
PATCH v2 1/3 libffi: fix headers location for host-libffi
From: Yegor Yefremov <[email protected]>
There is a special hook for target-libffi, that moves headers from /usr/lib/libffi-version/include to /usr/include. This patch adds the same procedure for host-libffi.
--- package/libffi/libffi.mk | 11 +++++++++++ 1 file changed, 11 insertions(+)
package/libffi/libffi.mk +11 -0
@@ -22,6 +22,17 @@ endef LIBFFI_POST_INSTALL_STAGING_HOOKS += LIBFFI_MOVE_STAGING_HEADERS +# Move the headers to the usual location, and adjust the .pc file +# accordingly. +define HOST_LIBFFI_MOVE_HOST_HEADERS + mv $(HOST_DIR)/usr/lib/libffi-$(LIBFFI_VERSION)/include/*.h $(HOST_DIR)/usr/include/ + $(SED) '/^includedir.*/d' -e '/^Cflags:.*/d' \ + $(HOST_DIR)/usr/lib/pkgconfig/libffi.pc + rm -rf $(HOST_DIR)/usr/lib/libffi-* +endef + +HOST_LIBFFI_POST_INSTALL_HOOKS += HOST_LIBFFI_MOVE_HOST_HEADERS + # Remove headers that are not at the usual location from the target define LIBFFI_REMOVE_TARGET_HEADERS $(RM) -rf $(TARGET_DIR)/usr/lib/libffi-$(LIBFFI_VERSION)
Signature · 1 line
2.1.4
PATCH v2 1/3 libffi: fix headers location for host-libffi
Hello,
Quoted 19 lines · yegorslists at googlemail.com
From: Yegor Yefremov <[email protected]>
There is a special hook for target-libffi, that moves headers from
/usr/lib/libffi-version/includeto/usr/include. This patch adds the same procedure for host-libffi.Signed-off-by Yegor Yefremov <[email protected]>--- package/libffi/libffi.mk | 11 +++++++++++ 1 file changed, 11 insertions(+)package/libffi/libffi.mk +0
-0@@ -22,6 +22,17 @@ endef LIBFFI_POST_INSTALL_STAGING_HOOKS += LIBFFI_MOVE_STAGING_HEADERS
package/libffi/libffi.mk endef · line 34
Quoted 10 lines
package/libffi/libffi.mk +10
-0+# Move the headers to the usual location, and adjust the .pc file +# accordingly. +define HOST_LIBFFI_MOVE_HOST_HEADERS + mv $(HOST_DIR)/usr/lib/libffi-$(LIBFFI_VERSION)/include/*.h $(HOST_DIR)/usr/include/ + $(SED) '/^includedir.*/d' -e '/^Cflags:.*/d' \ + $(HOST_DIR)/usr/lib/pkgconfig/libffi.pc + rm -rf $(HOST_DIR)/usr/lib/libffi-* +endef + +HOST_LIBFFI_POST_INSTALL_HOOKS += HOST_LIBFFI_MOVE_HOST_HEADERS
I've applied, but after doing some changes. First, the existing staging fixup had a mistake, it was removing the directory from $(TARGET_DIR), while it should have been removed from $(STAGING_DIR). I've fixed that in a preparation commit:
https://git.busybox.net/buildroot/commit/?id=2b73538920107b6c020ce5c4c8d9f7f672549bb1
Once this was fixed, your new code to take care of headers in $(HOST_DIR) was exactly the same as the one taking care of the headers in $(STAGING_DIR). So I've factorized the code in a common function, which takes as argument the base directory. Which gives:
https://git.busybox.net/buildroot/commit/?id=f41091176bb3e8da6cf78760b69f188e9868a259
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH v2 2/3 python-pycparser: new package
In reply to yegorslists at googlemail.comFrom: Yegor Yefremov <[email protected]>
pycparser requires C preprocessor in order to parse C related files, hence available only as host package.
---
Changes:
v2: remove target package variant
package/python-pycparser/python-pycparser.hash | 3 +++ package/python-pycparser/python-pycparser.mk | 14 ++++++++++++++ 2 files changed, 17 insertions(+) create mode 100644 package/python-pycparser/python-pycparser.hash create mode 100644 package/python-pycparser/python-pycparser.mk
package/python-pycparser/python-pycparser.hash new file +3 -0
@@ -0,0 +1,3 @@ +# md5 from https://pypi.python.org/pypi?:action=show_md5&digest=a2bc8d28c923b4fe2b2c3b4b51a4f935, sha256 locally computed +md5 a2bc8d28c923b4fe2b2c3b4b51a4f935 pycparser-2.14.tar.gz +sha256 7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73 pycparser-2.14.tar.gz
package/python-pycparser/python-pycparser.mk new file +14 -0
@@ -0,0 +1,14 @@ +################################################################################ +# +# python-pycparser +# +################################################################################ + +PYTHON_PYCPARSER_VERSION = 2.14 +PYTHON_PYCPARSER_SOURCE = pycparser-$(PYTHON_PYCPARSER_VERSION).tar.gz +PYTHON_PYCPARSER_SITE = https://pypi.python.org/packages/source/p/pycparser +PYTHON_PYCPARSER_SETUP_TYPE = setuptools +PYTHON_PYCPARSER_LICENSE = BSD-3c +PYTHON_PYCPARSER_LICENSE_FILES = LICENSE + +$(eval $(host-python-package))
Signature · 1 line
2.1.4
Thomas Petazzoni
AppliedPATCH v2 2/3 python-pycparser: new package
Dear yegorslists at googlemail.com,
Quoted 9 lines · yegorslists at googlemail.com
From: Yegor Yefremov <[email protected]>
pycparser requires C preprocessor in order to parse C related files, hence available only as host package.
Signed-off-by Yegor Yefremov <[email protected]>---
Changes:
v2: remove target package variant
Applied, thanks.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH v2 3/3 python-cffi: add host variant
In reply to yegorslists at googlemail.comFrom: Yegor Yefremov <[email protected]>
Host variant is needed to cross-compile CFFI based C library wrappers.
---
Changes:
v2: add host-libffi dependency
package/python-cffi/python-cffi.mk | 2 ++ 1 file changed, 2 insertions(+)
package/python-cffi/python-cffi.mk +2 -0
@@ -11,5 +11,7 @@ PYTHON_CFFI_SETUP_TYPE = setuptools PYTHON_CFFI_DEPENDENCIES = host-pkgconf libffi PYTHON_CFFI_LICENSE = MIT PYTHON_CFFI_LICENSE_FILES = LICENSE +HOST_PYTHON_CFFI_DEPENDENCIES = host-python-pycparser host-libffi $(eval $(python-package)) +$(eval $(host-python-package))
Signature · 1 line
2.1.4
PATCH v2 3/3 python-cffi: add host variant
Hello Thomas, Yann,
Quoted 17 lines · On Tue, Jan 5, 2016 at 8:17 AM, <[email protected]> wrote:
From: Yegor Yefremov <[email protected]>
Host variant is needed to cross-compile CFFI based C library wrappers.
Signed-off-by Yegor Yefremov <[email protected]>---
Changes:
v2: add host-libffi dependencypackage/python-cffi/python-cffi.mk | 2 ++ 1 file changed, 2 insertions(+)package/python-cffi/python-cffi.mk +0
-0
package/python-cffi/python-cffi.mk PYTHON_CFFI_SETUP_TYPE = setuptools
Quoted 8 lines
package/python-cffi/python-cffi.mk +2
-1@@ -11,5 +11,7 @@ PYTHON_CFFI_SETUP_TYPE = setuptools PYTHON_CFFI_DEPENDENCIES = host-pkgconf libffi PYTHON_CFFI_LICENSE = MIT PYTHON_CFFI_LICENSE_FILES = LICENSE +HOST_PYTHON_CFFI_DEPENDENCIES = host-python-pycparser host-libffi $(eval $(python-package)) +$(eval $(host-python-package)) --
I have now following theory regarding checking dependencies in target and host variant.
If you look at python-tornado's setup.py file [1], you see, that they say, install_requires will be checked only for setuptools. This is also the case for host-python-cffi. The target variant will be installed with --single-version-externally-managed option, that performs "old-style" install [2] [3], i.e. without checking install_requires.
That's my understanding for now.
[1] https://github.com/tornadoweb/tornado/blob/master/setup.py#L126
[2] https://pythonhosted.org/setuptools/setuptools.html
[3] https://bitbucket.org/pypa/setuptools/src/e517fced669d54bcaf3da2d8ca468d29120fedfc/setuptools/command/install.py?at=default&fileviewer=file-view-default#install.py-58
Yegor
PATCH v2 3/3 python-cffi: add host variant
Quoted 39 lines · Yegor Yefremov
Hello Thomas, Yann,
Quoted 17 lines · On Tue, Jan 5, 2016 at 8:17 AM, <[email protected]> wrote:
From: Yegor Yefremov <[email protected]>
Host variant is needed to cross-compile CFFI based C library wrappers.
Signed-off-by Yegor Yefremov <[email protected]>---
Changes:
v2: add host-libffi dependencypackage/python-cffi/python-cffi.mk | 2 ++ 1 file changed, 2 insertions(+)package/python-cffi/python-cffi.mk +0
-0package/python-cffi/python-cffi.mk PYTHON_CFFI_SETUP_TYPE = setuptools
Quoted 8 lines
package/python-cffi/python-cffi.mk +2
-1@@ -11,5 +11,7 @@ PYTHON_CFFI_SETUP_TYPE = setuptools PYTHON_CFFI_DEPENDENCIES = host-pkgconf libffi PYTHON_CFFI_LICENSE = MIT PYTHON_CFFI_LICENSE_FILES = LICENSE +HOST_PYTHON_CFFI_DEPENDENCIES = host-python-pycparser host-libffi $(eval $(python-package)) +$(eval $(host-python-package)) --I have now following theory regarding checking dependencies in target and host variant.
If you look at python-tornado's
setup.pyfile [1], you see, that they say,install_requireswill be checked only for setuptools. This is also the case for host-python-cffi. The target variant will be installed with --single-version-externally-managed option, that performs "old-style" install [2] [3], i.e. without checkinginstall_requires.
Quoted 5 lines
That's my understanding for now.
[1] https://github.com/tornadoweb/tornado/blob/master/setup.py#L126
[2] https://pythonhosted.org/setuptools/setuptools.html
[3] https://bitbucket.org/pypa/setuptools/src/e517fced669d54bcaf3da2d8ca468d29120fedfc/setuptools/command/install.py?at=default&fileviewer=file-view-default#install.py-58
This SO answer also points to this conclusion:
http://stackoverflow.com/a/22179371/1113139
Yegor
PATCH v2 3/3 python-cffi: add host variant
In reply to yegorslists at googlemail.comHello,
Quoted 17 lines · yegorslists at googlemail.com
From: Yegor Yefremov <[email protected]>
Host variant is needed to cross-compile CFFI based C library wrappers.
Signed-off-by Yegor Yefremov <[email protected]>---
Changes:
v2: add host-libffi dependencypackage/python-cffi/python-cffi.mk | 2 ++ 1 file changed, 2 insertions(+)package/python-cffi/python-cffi.mk +0
-0
package/python-cffi/python-cffi.mk PYTHON_CFFI_SETUP_TYPE = setuptools · line 17
Quoted 7 lines
package/python-cffi/python-cffi.mk +2
-0@@ -11,5 +11,7 @@ PYTHON_CFFI_SETUP_TYPE = setuptools PYTHON_CFFI_DEPENDENCIES = host-pkgconf libffi PYTHON_CFFI_LICENSE = MIT PYTHON_CFFI_LICENSE_FILES = LICENSE +HOST_PYTHON_CFFI_DEPENDENCIES = host-python-pycparser host-libffi $(eval $(python-package)) +$(eval $(host-python-package))
This wasn't building properly here, because python-cffi uses pkg-config to get the flags to link with libffi, and by default our pkg-config returns results that are valid to build stuff for the target, not for the host.
So, I've fixed that by passing a bunch of pkg-config environment variables when building the host package.
Also, you were missing the host-pkgconf dependency for the dependencies of host variant.
Committed with those issues fixed.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com