package/python-numpy: fixup numpy distutils for cross compilation
PATCH v4 1/2 package/python-numpy: fixup numpy distutils for cross compilation
Fix problems using the numpy distutils extension for cross compilation, so that linking with npymath library will use target library when building target packages.
--- .../0001-NPY_PKG_CONFIG_PATH.patch | 85 +++++++++++++++++++ package/python-numpy/python-numpy.mk | 21 +++++ 2 files changed, 106 insertions(+) create mode 100644 package/python-numpy/0001-NPY_PKG_CONFIG_PATH.patch
package/python-numpy/0001-NPY_PKG_CONFIG_PATH.patch new file +85 -0
@@ -0,0 +1,85 @@ +commit 153fc148eec60e5cbec0e80617f75a3a5dd2a3f8 +Author: Esben Haabendal <[email protected]> +Date: Thu Sep 12 21:59:58 2019 +0200 + + ENH: Allow NPY_PKG_CONFIG_PATH environment variable override + + Allow overriding npy-pkg-config directory using the NPY_PKG_CONFIG_PATH + environment variable, making it easier to use numpy in cross-compilation + setups. + +Upstream-Status: Accepted (scheduled for 1.18.0) +Signed-off-by: Esben Haabendal <[email protected]> + +diff --git a/numpy/distutils/misc_util.py b/numpy/distutils/misc_util.py +index 1e10e92fdadf..0eaaeb7364d4 100644 +--- a/numpy/distutils/misc_util.py ++++ b/numpy/distutils/misc_util.py +@@ -1687,6 +1687,43 @@ class Configuration(object): + + and will be installed as foo.ini in the 'lib' subpath. + ++ Cross-compilation ++ ----------------- ++ When cross-compiling with numpy distutils, it might be necessary to ++ use modified npy-pkg-config files. Using the default/generated files ++ will link with the host libraries (i.e. libnpymath.a). For ++ cross-compilation you of-course need to link with target libraries, ++ while using the host Python installation. ++ ++ You can copy out the numpy/core/lib/npy-pkg-config directory, add a ++ pkgdir value to the .ini files and set NPY_PKG_CONFIG_PATH environment ++ variable to point to the directory with the modified npy-pkg-config ++ files. ++ ++ Example npymath.ini modified for cross-compilation:: ++ ++ [meta] ++ Name=npymath ++ Description=Portable, core math library implementing C99 standard ++ Version=0.1 ++ ++ [variables] ++ pkgname=numpy.core ++ pkgdir=/build/arm-linux-gnueabi/sysroot/usr/lib/python3.7/site-packages/numpy/core ++ prefix=${pkgdir} ++ libdir=${prefix}/lib ++ includedir=${prefix}/include ++ ++ [default] ++ Libs=-L${libdir} -lnpymath ++ Cflags=-I${includedir} ++ Requires=mlib ++ ++ [msvc] ++ Libs=/LIBPATH:${libdir} npymath.lib ++ Cflags=/INCLUDE:${includedir} ++ Requires=mlib ++ + """ + if subst_dict is None: + subst_dict = {} +@@ -2092,9 +2129,22 @@ def get_numpy_include_dirs(): + return include_dirs + + def get_npy_pkg_dir(): +- """Return the path where to find the npy-pkg-config directory.""" ++ """Return the path where to find the npy-pkg-config directory. ++ ++ If the NPY_PKG_CONFIG_PATH environment variable is set, the value of that ++ is returned. Otherwise, a path inside the location of the numpy module is ++ returned. ++ ++ The NPY_PKG_CONFIG_PATH can be useful when cross-compiling, maintaining ++ customized npy-pkg-config .ini files for the cross-compilation ++ environment, and using them when cross-compiling. ++ ++ """ + # XXX: import here for bootstrapping reasons + import numpy ++ d = os.environ.get('NPY_PKG_CONFIG_PATH') ++ if d is not None: ++ return d + d = os.path.join(os.path.dirname(numpy.__file__), + 'core', 'lib', 'npy-pkg-config') + return d
package/python-numpy/python-numpy.mk +21 -0
@@ -30,6 +30,27 @@ define PYTHON_NUMPY_CONFIGURE_CMDS echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg endef +# The numpy distutils extensions are not very cross friendly. It comes with +# it's own pkg-config look-alike, which we are patching to allow overriding +# where it locates the configuration files. This allows us to use fixed up +# target configuration files, which we make sure includes full path to the +# target staging area, so that when building for target, we actually use the +# target libraries. Without this, target builds using numpy distutils +# extensions (such as fx. python-scipy) will use the host libraries, which +# obviously will not work. +ifeq ($(BR2_PACKAGE_PYTHON3),y) +PYTHON_NUMPY_STAGING_DIR = $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/numpy +else +PYTHON_NUMPY_STAGING_DIR = $(STAGING_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/numpy +endif +PYTHON_NUMPY_NPY_PKG_CONFIG_PATH=$(PYTHON_NUMPY_STAGING_DIR)/core/lib/npy-pkg-config +define PYTHON_NUMPY_FIXUP_NPY_PKG_CONFIG_FILES + sed -e '/^pkgdir=/d' \ + -e '/^prefix=/i pkgdir=$(PYTHON_NUMPY_STAGING_DIR)/core' \ + -i $(PYTHON_NUMPY_NPY_PKG_CONFIG_PATH)/npymath.ini +endef +PYTHON_NUMPY_POST_INSTALL_STAGING_HOOKS += PYTHON_NUMPY_FIXUP_NPY_PKG_CONFIG_FILES + # Some package may include few headers from NumPy, so let's install it # in the staging area. PYTHON_NUMPY_INSTALL_STAGING = YES
Signature · 1 line
2.23.0
PATCH v4 2/2 package/python-scipy: new package
See http://scipy.github.io/devdocs/building/linux.html
for SciPy build requirements:
* BLAS and LAPACK libraries
* C and Fortran compilers
* Python header files
* Cython compiler
For now, clapack package is used to provide both BLAS and LAPACK libraries, as that is what is supported by the current python-numpy recipe which python-scipy depends on.
With additional work, it should be possible to improve both python-numpy and python-scipy to support other BLAS and LAPACK library implementations, but it is probably best to keep such changes aligned.
A site.cfg file is used to configure numpy distutils extension to find include files and libraries in stagging.
Note, host-python-numpy is needed in order to use numpy distutils extension from it.
--- DEVELOPERS | 1 + package/Config.in | 1 + package/python-scipy/Config.in | 24 ++++++++++++++++ package/python-scipy/python-scipy.hash | 9 ++++++ package/python-scipy/python-scipy.mk | 40 ++++++++++++++++++++++++++ 5 files changed, 75 insertions(+) create mode 100644 package/python-scipy/Config.in create mode 100644 package/python-scipy/python-scipy.hash create mode 100644 package/python-scipy/python-scipy.mk
DEVELOPERS +1 -0
@@ -724,6 +724,7 @@ F: package/szip/ N: Esben Haabendal <[email protected]> F: boot/gummiboot/ F: package/python-kiwisolver/ +F: package/python-scipy/ N: Etienne Carriere <[email protected]> F: boot/optee-os/
package/Config.in +1 -0
@@ -1056,6 +1056,7 @@ menu "External python modules" source "package/python-scapy/Config.in" source "package/python-scapy3k/Config.in" source "package/python-schedule/Config.in" + source "package/python-scipy/Config.in" source "package/python-sdnotify/Config.in" source "package/python-secretstorage/Config.in" source "package/python-see/Config.in"
package/python-scipy/Config.in new file +24 -0
@@ -0,0 +1,24 @@ +config BR2_PACKAGE_PYTHON_SCIPY + bool "python-scipy" + depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS + depends on !BR2_powerpc || BR2_TOOLCHAIN_USES_GLIBC # clapack + depends on !BR2_m68k_cf # clapack + depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL # python-numpy + depends on BR2_TOOLCHAIN_HAS_FORTRAN + depends on BR2_PACKAGE_PYTHON3 + select BR2_PACKAGE_HOST_PYTHON_CYTHON + select BR2_PACKAGE_PYTHON_NUMPY + select BR2_PACKAGE_CLAPACK + help + The SciPy library is one of the core packages that make up the SciPy + stack. It provides many user-friendly and efficient numerical + routines such as routines for numerical integration, interpolation, + optimization, linear algebra and statistics. + + https://www.scipy.org/scipylib/ + +comment "python-scipy needs a toolchain w/ fortran" + depends on !BR2_TOOLCHAIN_HAS_FORTRAN + +comment "python-scipy needs glibc or musl toolchain" + depends on !BR2_TOOLCHAIN_USES_GLIBC && !BR2_TOOLCHAIN_USES_MUSL
package/python-scipy/python-scipy.hash new file +9 -0
@@ -0,0 +1,9 @@ +# Locally generated +sha256 4ea68de2840cf7d35c58464412d21f6f154cab7fba610c11002603ee0b9e2372 python-scipy-1.3.1.tar.gz +sha256 904942f7df1dc0f4a48af8d72405854fd96497bd1fe9ddb2c69d1797c22dfcd7 LICENSE.txt +sha256 37e64a498894ac7c3b070023e3689e954a8ecf8a23b90968d09a455f1b4f7b35 scipy/linalg/src/lapack_deprecations/LICENSE +sha256 606209a000716c5f66e33e180ce08434b96ed17db4975ab9723c6b5fbcc89609 scipy/ndimage/LICENSE.txt +sha256 3df9207af2fdb861af0ae3b22026f163b9bcfab4e525dc4943afe2ffa3f77624 scipy/optimize/tnc/LICENSE +sha256 f0cedf52503b2d42b83411a0a16e6fefac346dfad8fddc66f50050150123470c scipy/sparse/linalg/dsolve/SuperLU/License.txt +sha256 0926566f9f406178d1214f8cc796e166b1213dd7c05e0c5b461a8b8ac9e50bbe scipy/sparse/linalg/eigen/arpack/ARPACK/COPYING +sha256 51a5a08d537b34febb851c761496ead461f90b6c3c46a5248780d63870124e9c scipy/spatial/qhull_src/COPYING.txt
package/python-scipy/python-scipy.mk new file +40 -0
@@ -0,0 +1,40 @@ +################################################################################ +# +# python-scipy +# +################################################################################ + +PYTHON_SCIPY_VERSION = 1.3.1 +PYTHON_SCIPY_SITE = $(call github,scipy,scipy,v$(PYTHON_SCIPY_VERSION)) +PYTHON_SCIPY_LICENSE = BSD-3-Clause, BSD-2-Clause, BSD, BSD-Style, \ + Apache-2.0, MIT +PYTHON_SCIPY_LICENSE_FILES = \ + LICENSE.txt \ + scipy/linalg/src/lapack_deprecations/LICENSE \ + scipy/ndimage/LICENSE.txt \ + scipy/optimize/tnc/LICENSE \ + scipy/sparse/linalg/dsolve/SuperLU/License.txt \ + scipy/sparse/linalg/eigen/arpack/ARPACK/COPYING \ + scipy/spatial/qhull_src/COPYING.txt +PYTHON_SCIPY_SETUP_TYPE = setuptools +PYTHON_SCIPY_DEPENDENCIES += host-python-cython host-python-numpy python-numpy \ + clapack + +PYTHON_SCIPY_BUILD_OPTS = config_fc --fcompiler=gnu95 +PYTHON_SCIPY_ENV += F90=$(TARGET_CROSS)gfortran F77=$(TARGET_CROSS)gfortran + +# Provide system configuration options to numpy distutils extenions, telling +# to find all include files and libraries in staging directory. +define PYTHON_SCIPY_CONFIGURE_CMDS + -rm -f $(@D)/site.cfg + echo "[DEFAULT]" >> $(@D)/site.cfg + echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg + echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg +endef + +# Use the target numpy pkg-config configuration files modified for target +# cross-compilation. Without this, numpy distutils will cause the linker to +# link with host libnpymath.a. +PYTHON_SCIPY_ENV += NPY_PKG_CONFIG_PATH=$(PYTHON_NUMPY_NPY_PKG_CONFIG_PATH) + +$(eval $(python-package))
Signature · 1 line
2.23.0