Skip to contentSkip to conversation

python-scipy package

PATCH New Esben Haabendal 5 participants 15 messages Sep 20, 2019 – Sep 23, 2019

lore
Share & tools
lore
#0 lore raw

v2 cover python-scipy package

Add new python-scipy package and fixup python-numpy so python-scipy can be cross-compiled.

Changed in v2:

* Added mandatory and lovely comment header in python-scipy.mk file

Esben Haabendal (3):
package/python-numpy: fix setup type package/python-numpy: fixup numpy distutils for cross compilation package/python-scipy: new package

 DEVELOPERS                                    |  1 +
 package/Config.in                             |  1 +
 .../0001-NPY_PKG_CONFIG_PATH.patch            | 84 +++++++++++++++++++
 package/python-numpy/python-numpy.mk          | 25 +++++-
 package/python-scipy/Config.in                | 21 +++++
 package/python-scipy/python-scipy.hash        |  9 ++
 package/python-scipy/python-scipy.mk          | 37 ++++++++
 7 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 package/python-numpy/0001-NPY_PKG_CONFIG_PATH.patch
 create mode 100644 package/python-scipy/Config.in
 create mode 100644 package/python-scipy/python-scipy.hash
 create mode 100644 package/python-scipy/python-scipy.mk
Signature · 1 line
2.23.0
#1 lore raw

PATCH v2 1/3 package/python-numpy: fix setup type

The numpy package is using distutils, not setuptools.

---
 package/python-numpy/python-numpy.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/python-numpy/python-numpy.mk +1 -1
index 3b474efa6e6c..f1f6f8cba407 100644
@@ -13,7 +13,7 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
 			numpy/linalg/lapack_lite/LICENSE.txt \
 			tools/npy_tempita/license.txt \
 			numpy/core/src/multiarray/dragon4.c
-PYTHON_NUMPY_SETUP_TYPE = setuptools
+PYTHON_NUMPY_SETUP_TYPE = distutils
 
 ifeq ($(BR2_PACKAGE_CLAPACK),y)
 PYTHON_NUMPY_DEPENDENCIES += clapack
Signature · 1 line
2.23.0
#2 lore raw

PATCH v2 1/3 package/python-numpy: fix setup type

Hi Esben,

Quoted 2 lines · Esben Haabendal

The numpy package is using distutils, not setuptools.

Are you sure about this? See
https://github.com/numpy/numpy/blob/master/numpy/distutils/core.py

Yegor

Quoted 21 lines
---
 package/python-numpy/python-numpy.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/python-numpy/python-numpy.mk +1 -2
index 3b474efa6e6c..f1f6f8cba407 100644
@@ -13,7 +13,7 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
                        numpy/linalg/lapack_lite/LICENSE.txt \
                        tools/npy_tempita/license.txt \
                        numpy/core/src/multiarray/dragon4.c
-PYTHON_NUMPY_SETUP_TYPE = setuptools
+PYTHON_NUMPY_SETUP_TYPE = distutils
 
 ifeq ($(BR2_PACKAGE_CLAPACK),y)
 PYTHON_NUMPY_DEPENDENCIES += clapack
--

2.23.0

#4 lore raw

PATCH v2 1/3 package/python-numpy: fix setup type

Quoted 6 lines · Esben Haabendal

Hi Yegor

Quoted 2 lines · Yegor Yefremov
Quoted 1 line · Esben Haabendal
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

The numpy package is using distutils, not setuptools.

Are you sure about this? See
https://github.com/numpy/numpy/blob/master/numpy/distutils/core.py

https://github.com/numpy/numpy/blob/v1.16.4/setup.py#L392

Looks like a setuptools package to me.

Exactly, and that's how it is defined in Buildroot now. Changing it to
distutils would be wrong, no?

Regards,
Arnout

#5 lore raw

PATCH v2 1/3 package/python-numpy: fix setup type

Quoted 12 lines · Arnout Vandecappelle
Quoted 11 lines · Esben Haabendal

Hi Yegor

Quoted 6 lines · Yegor Yefremov
Quoted 2 lines · Esben Haabendal

The numpy package is using distutils, not setuptools.

Are you sure about this? See
https://github.com/numpy/numpy/blob/master/numpy/distutils/core.py

Quoted 6 lines
Quoted 3 lines

https://github.com/numpy/numpy/blob/v1.16.4/setup.py#L392

Looks like a setuptools package to me.

Exactly, and that's how it is defined in Buildroot now. Changing it to
distutils would be wrong, no?

You are absolutely right. EOUTOFCOFFEE.

/Esben

#6 lore raw

PATCH v2 2/3 package/python-numpy: fixup numpy distutils for cross compilation

In reply to Esben Haabendal

Fix problems using the numpy distutils extension for cross compilation, so that linking with npymath library will use target library when building target packages.

---
 .../0001-NPY_PKG_CONFIG_PATH.patch            | 84 +++++++++++++++++++
 package/python-numpy/python-numpy.mk          | 23 +++++
 2 files changed, 107 insertions(+)
 create mode 100644 package/python-numpy/0001-NPY_PKG_CONFIG_PATH.patch
package/python-numpy/0001-NPY_PKG_CONFIG_PATH.patch new file +84 -0
new file mode 100644
index 000000000000..cedf5ecc90e4
@@ -0,0 +1,84 @@
+commit 153fc148eec60e5cbec0e80617f75a3a5dd2a3f8
+Author: Esben Haabendal <[email protected]>
+Date:   Thu Sep 12 21:59:58 2019 +0200
+
+    ENH: Allow NPY_PKG_CONFIG_PATH environment variable override
+    
+    Allow overriding npy-pkg-config directory using the NPY_PKG_CONFIG_PATH
+    environment variable, making it easier to use numpy in cross-compilation
+    setups.
+
+Upstream-Status: Accepted (scheduled for 1.18.0)
+
+diff --git a/numpy/distutils/misc_util.py b/numpy/distutils/misc_util.py
+index 1e10e92fdadf..0eaaeb7364d4 100644
+--- a/numpy/distutils/misc_util.py
++++ b/numpy/distutils/misc_util.py
+@@ -1687,6 +1687,43 @@ class Configuration(object):
+ 
+         and will be installed as foo.ini in the 'lib' subpath.
+ 
++        Cross-compilation
++        -----------------
++        When cross-compiling with numpy distutils, it might be necessary to
++        use modified npy-pkg-config files.  Using the default/generated files
++        will link with the host libraries (i.e. libnpymath.a).  For
++        cross-compilation you of-course need to link with target libraries,
++        while using the host Python installation.
++
++        You can copy out the numpy/core/lib/npy-pkg-config directory, add a
++        pkgdir value to the .ini files and set NPY_PKG_CONFIG_PATH environment
++        variable to point to the directory with the modified npy-pkg-config
++        files.
++
++        Example npymath.ini modified for cross-compilation::
++
++            [meta]
++            Name=npymath
++            Description=Portable, core math library implementing C99 standard
++            Version=0.1
++
++            [variables]
++            pkgname=numpy.core
++            pkgdir=/build/arm-linux-gnueabi/sysroot/usr/lib/python3.7/site-packages/numpy/core
++            prefix=${pkgdir}
++            libdir=${prefix}/lib
++            includedir=${prefix}/include
++
++            [default]
++            Libs=-L${libdir} -lnpymath
++            Cflags=-I${includedir}
++            Requires=mlib
++
++            [msvc]
++            Libs=/LIBPATH:${libdir} npymath.lib
++            Cflags=/INCLUDE:${includedir}
++            Requires=mlib
++
+         """
+         if subst_dict is None:
+             subst_dict = {}
+@@ -2092,9 +2129,22 @@ def get_numpy_include_dirs():
+     return include_dirs
+ 
+ def get_npy_pkg_dir():
+-    """Return the path where to find the npy-pkg-config directory."""
++    """Return the path where to find the npy-pkg-config directory.
++
++    If the NPY_PKG_CONFIG_PATH environment variable is set, the value of that
++    is returned.  Otherwise, a path inside the location of the numpy module is
++    returned.
++
++    The NPY_PKG_CONFIG_PATH can be useful when cross-compiling, maintaining
++    customized npy-pkg-config .ini files for the cross-compilation
++    environment, and using them when cross-compiling.
++
++    """
+     # XXX: import here for bootstrapping reasons
+     import numpy
++    d = os.environ.get('NPY_PKG_CONFIG_PATH')
++    if d is not None:
++        return d
+     d = os.path.join(os.path.dirname(numpy.__file__),
+             'core', 'lib', 'npy-pkg-config')
+     return d
package/python-numpy/python-numpy.mk +23 -0
index f1f6f8cba407..3edb34a79ff7 100644
@@ -30,6 +30,29 @@ define PYTHON_NUMPY_CONFIGURE_CMDS
 	echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
 endef
 
+# The numpy distutils extensions are not very cross friendly.  It comes with
+# it's own pkg-config look-alike, which we are patching to allow overriding
+# where it locates the configuration files.  This allows us to use fixed up
+# target configuration files, which we make sure includes full path to the
+# target staging area, so that when building for target, we actually use the
+# target libraries.  Without this, target builds using numpy distutils
+# extensions (such as fx. python-scipy) will use the host libraries, which
+# obviously will not work.
+ifeq ($(BR2_PACKAGE_PYTHON3),y)
+PYTHON_NUMPY_STAGING_DIR = $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/numpy
+else
+PYTHON_NUMPY_STAGING_DIR = $(STAGING_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/numpy
+endif
+PYTHON_NUMPY_NPY_PKG_CONFIG_PATH=$(PYTHON_NUMPY_STAGING_DIR)/core/lib/npy-pkg-config
+define PYTHON_NUMPY_FIXUP_NPY_PKG_CONFIG_FILES
+	sed -i '/pkgdir=/d' \
+		$(PYTHON_NUMPY_NPY_PKG_CONFIG_PATH)/npymath.ini
+	awk -i inplace \
+		'/prefix=/ {print "pkgdir='$(PYTHON_NUMPY_STAGING_DIR)/core'"}1' \
+		$(PYTHON_NUMPY_NPY_PKG_CONFIG_PATH)/npymath.ini
+endef
+PYTHON_NUMPY_POST_INSTALL_STAGING_HOOKS += PYTHON_NUMPY_FIXUP_NPY_PKG_CONFIG_FILES
+
 # Some package may include few headers from NumPy, so let's install it
 # in the staging area.
 PYTHON_NUMPY_INSTALL_STAGING = YES
Signature · 1 line
2.23.0
#7 lore raw

PATCH v2 3/3 package/python-scipy: new package

In reply to Esben Haabendal
---
 DEVELOPERS                             |  1 +
 package/Config.in                      |  1 +
 package/python-scipy/Config.in         | 21 +++++++++++++++
 package/python-scipy/python-scipy.hash |  9 +++++++
 package/python-scipy/python-scipy.mk   | 37 ++++++++++++++++++++++++++
 5 files changed, 69 insertions(+)
 create mode 100644 package/python-scipy/Config.in
 create mode 100644 package/python-scipy/python-scipy.hash
 create mode 100644 package/python-scipy/python-scipy.mk
DEVELOPERS +1 -0
index 67a0fef0886d..975a197404bb 100644
@@ -723,6 +723,7 @@ F:	package/szip/
 N:	Esben Haabendal <[email protected]>
 F:	boot/gummiboot/
 F:	package/python-kiwisolver/
+F:	package/python-scipy/
 
 N:	Etienne Carriere <[email protected]>
 F:	boot/optee-os/
package/Config.in +1 -0
index dbf297f4df39..818a2abca591 100644
@@ -1056,6 +1056,7 @@ menu "External python modules"
 	source "package/python-scapy/Config.in"
 	source "package/python-scapy3k/Config.in"
 	source "package/python-schedule/Config.in"
+	source "package/python-scipy/Config.in"
 	source "package/python-sdnotify/Config.in"
 	source "package/python-secretstorage/Config.in"
 	source "package/python-see/Config.in"
package/python-scipy/Config.in new file +21 -0
new file mode 100644
index 000000000000..5b8192266843
@@ -0,0 +1,21 @@
+config BR2_PACKAGE_PYTHON_SCIPY
+	bool "python-scipy"
+	depends on BR2_PACKAGE_PYTHON3
+	depends on BR2_TOOLCHAIN_HAS_FORTRAN
+	depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_OPENBLAS_ARCH_SUPPORTS
+	select BR2_PACKAGE_HOST_PYTHON_CYTHON
+	select BR2_PACKAGE_HOST_PYTHON_NUMPY
+	select BR2_PACKAGE_PYTHON_NUMPY
+	select BR2_PACKAGE_OPENBLAS
+	select BR2_PACKAGE_CLAPACK
+	help
+	  The SciPy library is one of the core packages that make up the SciPy
+	  stack. It provides many user-friendly and efficient numerical
+	  routines such as routines for numerical integration, interpolation,
+	  optimization, linear algebra and statistics.
+
+	  https://www.scipy.org/scipylib/
+
+comment "python-scipy needs a toolchain w/ fortran"
+	depends on !BR2_TOOLCHAIN_HAS_FORTRAN
package/python-scipy/python-scipy.hash new file +9 -0
new file mode 100644
index 000000000000..6b0c3605a3b8
@@ -0,0 +1,9 @@
+# Locally generated
+sha256 4ea68de2840cf7d35c58464412d21f6f154cab7fba610c11002603ee0b9e2372  python-scipy-1.3.1.tar.gz
+sha256 904942f7df1dc0f4a48af8d72405854fd96497bd1fe9ddb2c69d1797c22dfcd7  LICENSE.txt
+sha256 37e64a498894ac7c3b070023e3689e954a8ecf8a23b90968d09a455f1b4f7b35  scipy/linalg/src/lapack_deprecations/LICENSE
+sha256 606209a000716c5f66e33e180ce08434b96ed17db4975ab9723c6b5fbcc89609  scipy/ndimage/LICENSE.txt
+sha256 3df9207af2fdb861af0ae3b22026f163b9bcfab4e525dc4943afe2ffa3f77624  scipy/optimize/tnc/LICENSE
+sha256 f0cedf52503b2d42b83411a0a16e6fefac346dfad8fddc66f50050150123470c  scipy/sparse/linalg/dsolve/SuperLU/License.txt
+sha256 0926566f9f406178d1214f8cc796e166b1213dd7c05e0c5b461a8b8ac9e50bbe  scipy/sparse/linalg/eigen/arpack/ARPACK/COPYING
+sha256 51a5a08d537b34febb851c761496ead461f90b6c3c46a5248780d63870124e9c  scipy/spatial/qhull_src/COPYING.txt
package/python-scipy/python-scipy.mk new file +37 -0
new file mode 100644
index 000000000000..74db98ed0ba0
@@ -0,0 +1,37 @@
+################################################################################
+#
+# python-scipy
+#
+################################################################################
+
+PYTHON_SCIPY_VERSION = 1.3.1
+PYTHON_SCIPY_SITE = $(call github,scipy,scipy,v$(PYTHON_SCIPY_VERSION))
+PYTHON_SCIPY_LICENSE = BSD-3-Clause, BSD-2-Clause, BSD, BSD-Style, PSF, \
+	Apache-2.0, MIT
+PYTHON_SCIPY_LICENSE_FILES = LICENSE.txt \
+	scipy/linalg/src/lapack_deprecations/LICENSE \
+	scipy/ndimage/LICENSE.txt \
+	scipy/optimize/tnc/LICENSE \
+	scipy/sparse/linalg/dsolve/SuperLU/License.txt \
+	scipy/sparse/linalg/eigen/arpack/ARPACK/COPYING \
+	scipy/spatial/qhull_src/COPYING.txt
+PYTHON_SCIPY_SETUP_TYPE = setuptools
+PYTHON_SCIPY_DEPENDENCIES += host-python-cython host-python-numpy python-numpy \
+	openblas clapack
+
+PYTHON_SCIPY_BUILD_OPTS = --fcompiler=gnu95
+PYTHON_SCIPY_ENV += F90=$(TARGET_CROSS)gfortran F77=$(TARGET_CROSS)gfortran
+
+define PYTHON_SCIPY_CONFIGURE_CMDS
+	-rm -f $(@D)/site.cfg
+	echo "[DEFAULT]" >> $(@D)/site.cfg
+	echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg
+	echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
+endef
+
+# Use the target numpy pkg-config configuration files modified for target
+# cross-compilation.  Without this, numpy distutils will cause the linker to
+# link with host libnpymath.a.
+PYTHON_SCIPY_ENV += NPY_PKG_CONFIG_PATH=$(PYTHON_NUMPY_NPY_PKG_CONFIG_PATH)
+
+$(eval $(python-package))
Signature · 1 line
2.23.0
#8 lore raw

PATCH v2 3/3 package/python-scipy: new package

Hi Esben,

There is a previous contribution to add python-scipy from Alexandre Payen:
http://patchwork.ozlabs.org/patch/1141144/

It would be great if you can work with Alexandre on python-scipy package.

Le 20/09/2019 ? 18:36, Esben Haabendal a ?crit?:

Quoted 41 lines
---
 DEVELOPERS                             |  1 +
 package/Config.in                      |  1 +
 package/python-scipy/Config.in         | 21 +++++++++++++++
 package/python-scipy/python-scipy.hash |  9 +++++++
 package/python-scipy/python-scipy.mk   | 37 ++++++++++++++++++++++++++
 5 files changed, 69 insertions(+)
 create mode 100644 package/python-scipy/Config.in
 create mode 100644 package/python-scipy/python-scipy.hash
 create mode 100644 package/python-scipy/python-scipy.mk
DEVELOPERS +1 -0
index 67a0fef0886d..975a197404bb 100644
@@ -723,6 +723,7 @@ F:	package/szip/
 N:	Esben Haabendal <[email protected]>
 F:	boot/gummiboot/
 F:	package/python-kiwisolver/
+F:	package/python-scipy/
 
 N:	Etienne Carriere <[email protected]>
 F:	boot/optee-os/
package/Config.in +1 -0
index dbf297f4df39..818a2abca591 100644
@@ -1056,6 +1056,7 @@ menu "External python modules"
 	source "package/python-scapy/Config.in"
 	source "package/python-scapy3k/Config.in"
 	source "package/python-schedule/Config.in"
+	source "package/python-scipy/Config.in"
 	source "package/python-sdnotify/Config.in"
 	source "package/python-secretstorage/Config.in"
 	source "package/python-see/Config.in"
package/python-scipy/Config.in new file +0 -0
new file mode 100644
index 000000000000..5b8192266843

package/python-scipy/Config.in line 8

Quoted 8 lines
package/python-scipy/Config.in new file +8 -0
@@ -0,0 +1,21 @@
+config BR2_PACKAGE_PYTHON_SCIPY
+	bool "python-scipy"
+	depends on BR2_PACKAGE_PYTHON3
+	depends on BR2_TOOLCHAIN_HAS_FORTRAN
+	depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_OPENBLAS_ARCH_SUPPORTS
+	select BR2_PACKAGE_HOST_PYTHON_CYTHON
+	select BR2_PACKAGE_HOST_PYTHON_NUMPY

There is no such option, the dependency is trigged by adding host-python-numpy in DEPENDENCIES;

package/python-scipy/Config.in line 11

Quoted 3 lines
package/python-scipy/Config.in new file +3 -0
+	select BR2_PACKAGE_PYTHON_NUMPY
+	select BR2_PACKAGE_OPENBLAS
+	select BR2_PACKAGE_CLAPACK

Python-scipy can be used with lapack instead of Clapack. Clapack is not maintained any more and really out of date.

OpenBlas is using a bundled version of Lapack when gFortran compiler is available. Clapack is an old f2c'ed version of lapack, so it can be used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding new packages using *blas libraries :-/

Best regards,
Romain

Quoted 69 lines
package/python-scipy/Config.in new file +10 -0
+	help
+	  The SciPy library is one of the core packages that make up the SciPy
+	  stack. It provides many user-friendly and efficient numerical
+	  routines such as routines for numerical integration, interpolation,
+	  optimization, linear algebra and statistics.
+
+	  https://www.scipy.org/scipylib/
+
+comment "python-scipy needs a toolchain w/ fortran"
+	depends on !BR2_TOOLCHAIN_HAS_FORTRAN
package/python-scipy/python-scipy.hash new file +9 -0
new file mode 100644
index 000000000000..6b0c3605a3b8
@@ -0,0 +1,9 @@
+# Locally generated
+sha256 4ea68de2840cf7d35c58464412d21f6f154cab7fba610c11002603ee0b9e2372  python-scipy-1.3.1.tar.gz
+sha256 904942f7df1dc0f4a48af8d72405854fd96497bd1fe9ddb2c69d1797c22dfcd7  LICENSE.txt
+sha256 37e64a498894ac7c3b070023e3689e954a8ecf8a23b90968d09a455f1b4f7b35  scipy/linalg/src/lapack_deprecations/LICENSE
+sha256 606209a000716c5f66e33e180ce08434b96ed17db4975ab9723c6b5fbcc89609  scipy/ndimage/LICENSE.txt
+sha256 3df9207af2fdb861af0ae3b22026f163b9bcfab4e525dc4943afe2ffa3f77624  scipy/optimize/tnc/LICENSE
+sha256 f0cedf52503b2d42b83411a0a16e6fefac346dfad8fddc66f50050150123470c  scipy/sparse/linalg/dsolve/SuperLU/License.txt
+sha256 0926566f9f406178d1214f8cc796e166b1213dd7c05e0c5b461a8b8ac9e50bbe  scipy/sparse/linalg/eigen/arpack/ARPACK/COPYING
+sha256 51a5a08d537b34febb851c761496ead461f90b6c3c46a5248780d63870124e9c  scipy/spatial/qhull_src/COPYING.txt
package/python-scipy/python-scipy.mk new file +37 -0
new file mode 100644
index 000000000000..74db98ed0ba0
@@ -0,0 +1,37 @@
+################################################################################
+#
+# python-scipy
+#
+################################################################################
+
+PYTHON_SCIPY_VERSION = 1.3.1
+PYTHON_SCIPY_SITE = $(call github,scipy,scipy,v$(PYTHON_SCIPY_VERSION))
+PYTHON_SCIPY_LICENSE = BSD-3-Clause, BSD-2-Clause, BSD, BSD-Style, PSF, \
+	Apache-2.0, MIT
+PYTHON_SCIPY_LICENSE_FILES = LICENSE.txt \
+	scipy/linalg/src/lapack_deprecations/LICENSE \
+	scipy/ndimage/LICENSE.txt \
+	scipy/optimize/tnc/LICENSE \
+	scipy/sparse/linalg/dsolve/SuperLU/License.txt \
+	scipy/sparse/linalg/eigen/arpack/ARPACK/COPYING \
+	scipy/spatial/qhull_src/COPYING.txt
+PYTHON_SCIPY_SETUP_TYPE = setuptools
+PYTHON_SCIPY_DEPENDENCIES += host-python-cython host-python-numpy python-numpy \
+	openblas clapack
+
+PYTHON_SCIPY_BUILD_OPTS = --fcompiler=gnu95
+PYTHON_SCIPY_ENV += F90=$(TARGET_CROSS)gfortran F77=$(TARGET_CROSS)gfortran
+
+define PYTHON_SCIPY_CONFIGURE_CMDS
+	-rm -f $(@D)/site.cfg
+	echo "[DEFAULT]" >> $(@D)/site.cfg
+	echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg
+	echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
+endef
+
+# Use the target numpy pkg-config configuration files modified for target
+# cross-compilation.  Without this, numpy distutils will cause the linker to
+# link with host libnpymath.a.
+PYTHON_SCIPY_ENV += NPY_PKG_CONFIG_PATH=$(PYTHON_NUMPY_NPY_PKG_CONFIG_PATH)
+
+$(eval $(python-package))
#9 lore raw

PATCH v2 3/3 package/python-scipy: new package

Hi Romain

Quoted 4 lines · Romain Naour

There is a previous contribution to add python-scipy from Alexandre Payen:
http://patchwork.ozlabs.org/patch/1141144/

It would be great if you can work with Alexandre on python-scipy package.

Sounds good.

Alexandre, do you have time to work on this?

Quoted 45 lines

Le 20/09/2019 ? 18:36, Esben Haabendal a ?crit?:

Quoted 44 lines
---
 DEVELOPERS                             |  1 +
 package/Config.in                      |  1 +
 package/python-scipy/Config.in         | 21 +++++++++++++++
 package/python-scipy/python-scipy.hash |  9 +++++++
 package/python-scipy/python-scipy.mk   | 37 ++++++++++++++++++++++++++
 5 files changed, 69 insertions(+)
 create mode 100644 package/python-scipy/Config.in
 create mode 100644 package/python-scipy/python-scipy.hash
 create mode 100644 package/python-scipy/python-scipy.mk
DEVELOPERS +1 -0
index 67a0fef0886d..975a197404bb 100644
@@ -723,6 +723,7 @@ F:	package/szip/
 N:	Esben Haabendal <[email protected]>
 F:	boot/gummiboot/
 F:	package/python-kiwisolver/
+F:	package/python-scipy/
 
 N:	Etienne Carriere <[email protected]>
 F:	boot/optee-os/
package/Config.in +1 -0
index dbf297f4df39..818a2abca591 100644
@@ -1056,6 +1056,7 @@ menu "External python modules"
 	source "package/python-scapy/Config.in"
 	source "package/python-scapy3k/Config.in"
 	source "package/python-schedule/Config.in"
+	source "package/python-scipy/Config.in"
 	source "package/python-sdnotify/Config.in"
 	source "package/python-secretstorage/Config.in"
 	source "package/python-see/Config.in"
package/python-scipy/Config.in new file +3 -0
new file mode 100644
index 000000000000..5b8192266843
@@ -0,0 +1,21 @@
+config BR2_PACKAGE_PYTHON_SCIPY
+	bool "python-scipy"
+	depends on BR2_PACKAGE_PYTHON3
Quoted 8 lines

package/python-scipy/Config.in line 8

Quoted 5 lines
package/python-scipy/Config.in new file +5 -0
+	depends on BR2_TOOLCHAIN_HAS_FORTRAN
+	depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_OPENBLAS_ARCH_SUPPORTS
+	select BR2_PACKAGE_HOST_PYTHON_CYTHON
+	select BR2_PACKAGE_HOST_PYTHON_NUMPY

There is no such option, the dependency is trigged by adding host-python-numpy in DEPENDENCIES;

Ah, I wasn't aware of how host-foo packages in DEPENDENCIES works. I will remove the bogus select.

Quoted 6 lines

package/python-scipy/Config.in line 11

Quoted 3 lines
package/python-scipy/Config.in new file +3 -0
+	select BR2_PACKAGE_PYTHON_NUMPY
+	select BR2_PACKAGE_OPENBLAS
+	select BR2_PACKAGE_CLAPACK

Python-scipy can be used with lapack instead of Clapack. Clapack is not maintained any more and really out of date.

Yes, but as we currently only supports clapack for python-numpy builds, It seems a bit odd to do it different for python-scipy.

What about adding python-scipy first, and then extending both python-numpy and python-scipy to support lapack as (an up-to-date) alternative to clapack afterwards?

Quoted 8 lines

OpenBlas is using a bundled version of Lapack when gFortran compiler is available. Clapack is an old f2c'ed version of lapack, so it can be used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding new packages using *blas libraries :-/

Ok, so where does this place this work on python-scipy for now? Is it a 100% no-go until *blas providers are reworked?

I think it would be a shame to block python-scipy, as it is pretty useful package, and the effort to take it in (looking at this patch series) is pretty minimal.

/Esben

#10 lore raw

PATCH v2 3/3 package/python-scipy: new package

Hi Esben.

Le lun. 23 sept. 2019 ? 09:05, Esben Haabendal <[email protected]> a ?crit :

Quoted 5 lines

Hi Romain

Quoted 1 line · Romain Naour

There is a previous contribution to add python-scipy from Alexandre

Quoted 9 lines

Payen:

Quoted 3 lines

http://patchwork.ozlabs.org/patch/1141144/

It would be great if you can work with Alexandre on python-scipy package.

Sounds good.

Alexandre, do you have time to work on this?

I have some time if you need yes.

Quoted 67 lines
Quoted 23 lines

Le 20/09/2019 ? 18:36, Esben Haabendal a ?crit :

Quoted 21 lines
---
 DEVELOPERS                             |  1 +
 package/Config.in                      |  1 +
 package/python-scipy/Config.in         | 21 +++++++++++++++
 package/python-scipy/python-scipy.hash |  9 +++++++
 package/python-scipy/python-scipy.mk   | 37 ++++++++++++++++++++++++++
 5 files changed, 69 insertions(+)
 create mode 100644 package/python-scipy/Config.in
 create mode 100644 package/python-scipy/python-scipy.hash
 create mode 100644 package/python-scipy/python-scipy.mk
DEVELOPERS +1 -0
index 67a0fef0886d..975a197404bb 100644
@@ -723,6 +723,7 @@ F:       package/szip/
 N:  Esben Haabendal <[email protected]>
 F:  boot/gummiboot/
 F:  package/python-kiwisolver/
+F:  package/python-scipy/
 
Quoted 15 lines

Review

Quoted 15 lines
DEVELOPERS +0 -0
 N:  Etienne Carriere <[email protected]>
 F:  boot/optee-os/
package/Config.in +1 -0
index dbf297f4df39..818a2abca591 100644
@@ -1056,6 +1056,7 @@ menu "External python modules"
     source "package/python-scapy/Config.in"
     source "package/python-scapy3k/Config.in"
     source "package/python-schedule/Config.in"
+    source "package/python-scipy/Config.in"
     source "package/python-sdnotify/Config.in"
     source "package/python-secretstorage/Config.in"
     source "package/python-see/Config.in"
(quoted diff) +0 -0

b/package/python-scipy/Config.in

Quoted 7 lines
Quoted 7 lines

new file mode 100644
index 000000000000..5b8192266843

package/python-scipy/Config.in new file +2 -0
@@ -0,0 +1,21 @@
+config BR2_PACKAGE_PYTHON_SCIPY
+    bool "python-scipy"
Quoted 8 lines

package/python-scipy/Config.in line 8

Quoted 6 lines
package/python-scipy/Config.in new file +6 -0
+    depends on BR2_PACKAGE_PYTHON3
+    depends on BR2_TOOLCHAIN_HAS_FORTRAN
+    depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS
+    depends on BR2_PACKAGE_OPENBLAS_ARCH_SUPPORTS
+    select BR2_PACKAGE_HOST_PYTHON_CYTHON
+    select BR2_PACKAGE_HOST_PYTHON_NUMPY

There is no such option, the dependency is trigged by adding

host-python-numpy

Quoted 1 line

in DEPENDENCIES;

Ah, I wasn't aware of how host-foo packages in DEPENDENCIES works. I will remove the bogus select.

Quoted 6 lines

package/python-scipy/Config.in line 11

Quoted 3 lines
package/python-scipy/Config.in new file +3 -0
+    select BR2_PACKAGE_PYTHON_NUMPY
+    select BR2_PACKAGE_OPENBLAS
+    select BR2_PACKAGE_CLAPACK

Python-scipy can be used with lapack instead of Clapack. Clapack is not maintained any more and really out of date.

Quoted 7 lines

Yes, but as we currently only supports clapack for python-numpy builds, It seems a bit odd to do it different for python-scipy.

What about adding python-scipy first, and then extending both python-numpy and python-scipy to support lapack as (an up-to-date) alternative to clapack afterwards?

This is exatcly what I did. I propose a virtual package for lapack/clapack. But after digging into lapack/clapack, we realise this is much harder than just adding a virtual package.

Quoted 6 lines
Quoted 6 lines

OpenBlas is using a bundled version of Lapack when gFortran compiler is available. Clapack is an old f2c'ed version of lapack, so it can be used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Quoted 7 lines
Quoted 3 lines

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding new packages using *blas libraries :-/

Ok, so where does this place this work on python-scipy for now? Is it a 100% no-go until *blas providers are reworked?

For me, yes, the first setp is to fix those blas providers. There is libflame[1], a lapack implementation and Blis[2], a blas implementation.
libflame includes a compatibility layer, lapack2flame, which includes a complete LAPACK implementation.
While BLIS exports a new BLAS-like API, it also includes a BLAS compatibility layer which gives application developers access to BLIS implementations via traditional BLAS routine calls.

This have to be test but I guess it could replace lapack/clapack and blas implementation.
I suggest you to read the previous work on lapack/clapack and blas[3].

Quoted 7 lines

I think it would be a shame to block python-scipy, as it is pretty useful package, and the effort to take it in (looking at this patch series) is pretty minimal.

/Esben

Cheers,
Alexandre.

[1] : https://github.com/flame/libflame
[2] : https://github.com/flame/blis
[3] :
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-1-5-package-python-numpy-fix-occasional-build-and-run-time-failure-with-lapack-td228787.html#a228788
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190923/c7880a74/attachment.html>

#11 lore raw

PATCH v2 3/3 package/python-scipy: new package

Hi Alexandre

Quoted 3 lines · Alexandre PAYEN
Quoted 2 lines

Alexandre, do you have time to work on this?

I have some time if you need yes.

Great :)

Quoted 10 lines
Quoted 10 lines
Quoted 6 lines

package/python-scipy/Config.in line 11

Quoted 3 lines
package/python-scipy/Config.in new file +3 -0
+    select BR2_PACKAGE_PYTHON_NUMPY
+    select BR2_PACKAGE_OPENBLAS
+    select BR2_PACKAGE_CLAPACK

Python-scipy can be used with lapack instead of Clapack. Clapack is not maintained any more and really out of date.

Yes, but as we currently only supports clapack for python-numpy builds, It seems a bit odd to do it different for python-scipy.

Quoted 7 lines
Quoted 4 lines

What about adding python-scipy first, and then extending both python-numpy and python-scipy to support lapack as (an up-to-date) alternative to clapack afterwards?

This is exatcly what I did. I propose a virtual package for lapack/clapack. But after digging into lapack/clapack, we realise this is much harder than just adding a virtual package.

What is harder, adding python-scipy first?

If so, I think you might been hindered by a cross-compilation unfriendly mechanism in numpy distutils, which I have got a fix into numpy for. Please take a look at
https://github.com/numpy/numpy/pull/14495/commits/153fc148eec60e5cbec0e80617f75a3a5dd2a3f8

and how I use that in my python-scipy recipe sent to this list.

With that, I think the python-scipy recipe looks pretty simple and innocent, and I don't think that merging it should be a problem for reworking lapack/clpack/*blas later on.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

OpenBlas is using a bundled version of Lapack when gFortran compiler is available. Clapack is an old f2c'ed version of lapack, so it can be used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding new packages using *blas libraries :-/

Ok, so where does this place this work on python-scipy for now? Is it a 100% no-go until *blas providers are reworked?

For me, yes, the first setp is to fix those blas providers.

Ok, but wouldn't it help having the python-scipy in here to have a good use-case for working with all this?

Quoted 11 lines

There is libflame[1], a lapack implementation and Blis[2], a blas implementation.
libflame includes a compatibility layer, lapack2flame, which includes a complete LAPACK implementation.
While BLIS exports a new BLAS-like API, it also includes a BLAS compatibility layer which gives application developers access to BLIS implementations via traditional BLAS routine calls.

This have to be test but I guess it could replace lapack/clapack and blas implementation.
I suggest you to read the previous work on lapack/clapack and blas[3].

I am not at all a lapack user/developer, just trying to build something with it, so please bear with me.

You write that the python-scipy requires lapack (and thus implicitl that it does not work with clapack). Why is that? It seems to WORKSFORME, so what am I missing?

Quoted 6 lines
Quoted 3 lines

I think it would be a shame to block python-scipy, as it is pretty useful package, and the effort to take it in (looking at this patch series) is pretty minimal.

[3] :
http://buildroot-busybox.2317881.n4.nabble.com/PATCH-1-5-package-python-numpy-fix-occasional-build-and-run-time-failure-with-lapack-td228787.html#a228788

/Esben

#12 lore raw

PATCH v2 3/3 package/python-scipy: new package

Le lun. 23 sept. 2019 ? 10:51, Esben Haabendal <[email protected]> a ?crit :

Quoted 24 lines

Hi Alexandre

Quoted 3 lines · Alexandre PAYEN
Quoted 2 lines

Alexandre, do you have time to work on this?

I have some time if you need yes.

Great :)

Quoted 14 lines
Quoted 14 lines
Quoted 6 lines

package/python-scipy/Config.in line 11

Quoted 3 lines
package/python-scipy/Config.in new file +3 -0
+    select BR2_PACKAGE_PYTHON_NUMPY
+    select BR2_PACKAGE_OPENBLAS
+    select BR2_PACKAGE_CLAPACK

Python-scipy can be used with lapack instead of Clapack. Clapack is not maintained any more and really out of date.

Yes, but as we currently only supports clapack for python-numpy builds, It seems a bit odd to do it different for python-scipy.

What about adding python-scipy first, and then extending both python-numpy and python-scipy to support lapack as (an up-to-date) alternative to clapack afterwards?

Quoted 8 lines
Quoted 1 line

This is exatcly what I did. I propose a virtual package for

lapack/clapack.

Quoted 1 line

But after digging into lapack/clapack, we realise this is much harder

than

Quoted 1 line

just adding a virtual package.

What is harder, adding python-scipy first?

No, lapack/clapack and blas.

Quoted 5 lines

If so, I think you might been hindered by a cross-compilation unfriendly mechanism in numpy distutils, which I have got a fix into numpy for. Please take a look at

Quoted 7 lines

https://github.com/numpy/numpy/pull/14495/commits/153fc148eec60e5cbec0e80617f75a3a5dd2a3f8

and how I use that in my python-scipy recipe sent to this list.

With that, I think the python-scipy recipe looks pretty simple and innocent, and I don't think that merging it should be a problem for reworking lapack/clpack/*blas later on.

Yes, there was a cross-compilation problem. But we fix it poorly, I assume your commit do the job.
I think adding python-scipy is not relevant because we don't understand lapack/clapack and blas properly.
We need to fix blas/lapack before adding packages which uses it.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

OpenBlas is using a bundled version of Lapack when gFortran compiler is available. Clapack is an old f2c'ed version of lapack, so it can be used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding new packages using *blas libraries :-/

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

Ok, so where does this place this work on python-scipy for now? Is it a 100% no-go until *blas providers are reworked?

For me, yes, the first setp is to fix those blas providers.

Ok, but wouldn't it help having the python-scipy in here to have a good use-case for working with all this?

Yes, why not.
But python-numpy is already present. It's help testing lapack with or without complex support.

Quoted 9 lines
Quoted 9 lines

There is libflame[1], a lapack implementation and Blis[2], a blas implementation.
libflame includes a compatibility layer, lapack2flame, which includes a complete LAPACK implementation.
While BLIS exports a new BLAS-like API, it also includes a BLAS compatibility layer which gives application developers access to BLIS implementations via traditional BLAS routine calls.

Quoted 7 lines
Quoted 3 lines

This have to be test but I guess it could replace lapack/clapack and blas implementation.
I suggest you to read the previous work on lapack/clapack and blas[3].

I am not at all a lapack user/developer, just trying to build something with it, so please bear with me.

No one really use lapack, python-scipy use it, by building python-scipy, you kind of become a lapack user.

Quoted 5 lines

You write that the python-scipy requires lapack (and thus implicitl that it does not work with clapack). Why is that? It seems to WORKSFORME, so what am I missing?

No, clapack is just a C implementation of lapack. It had been generated in C from fortran code with f2c.
Clapack is not maintained any more as lapack but there are still old. Also, lapack need a fortran compiler as python-numpy/python-scipy then you should use lapack instead of clapack.

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines

I think it would be a shame to block python-scipy, as it is pretty useful package, and the effort to take it in (looking at this patch series) is pretty minimal.

Quoted 7 lines
Quoted 2 lines

[3] :

http://buildroot-busybox.2317881.n4.nabble.com/PATCH-1-5-package-python-numpy-fix-occasional-build-and-run-time-failure-with-lapack-td228787.html#a228788

/Esben

Alexandre.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190923/58ecd5bd/attachment.html>

#13 lore raw

PATCH v2 3/3 package/python-scipy: new package

Hi Alexandre

Quoted 11 lines · Alexandre PAYEN

Le lun. 23 sept. 2019 ? 10:51, Esben Haabendal <[email protected]> a ?crit :

Quoted 8 lines
Quoted 3 lines · Alexandre PAYEN

This is exatcly what I did. I propose a virtual package for lapack/clapack. But after digging into lapack/clapack, we realise this is much harder than just adding a virtual package.

What is harder, adding python-scipy first?

No, lapack/clapack and blas.

Ok. So while delaying addition of python-scipy with a relatively simple virtual package change, we now have a much harder task at hand.

Quoted 6 lines
Quoted 6 lines

If so, I think you might been hindered by a cross-compilation unfriendly mechanism in numpy distutils, which I have got a fix into numpy for. Please take a look at

https://github.com/numpy/numpy/pull/14495/commits/153fc148eec60e5cbec0e80617f75a3a5dd2a3f8

Quoted 8 lines
Quoted 5 lines

and how I use that in my python-scipy recipe sent to this list.

With that, I think the python-scipy recipe looks pretty simple and innocent, and I don't think that merging it should be a problem for reworking lapack/clpack/*blas later on.

Yes, there was a cross-compilation problem. But we fix it poorly, I assume your commit do the job.

I believe it does.

Quoted 2 lines

I think adding python-scipy is not relevant because we don't understand lapack/clapack and blas properly.

Well, addition of python-scipy is relevant for me, as I need to have support for it. It is a firm product requirement, so the lack of it requires me to carry it in an overlay or to maintain patches for it.

Quoted 1 line

We need to fix blas/lapack before adding packages which uses it.

It might be old, but clapack seems to do the job for python-scipy. So I don't see that we *need* to do something to blas/lapack before adding python-scipy, but it definitely sounds like it would be good to do something about it, with or without python-scipy.

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 8 lines

OpenBlas is using a bundled version of Lapack when gFortran compiler is available. Clapack is an old f2c'ed version of lapack, so it can be used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding new packages using *blas libraries :-/

Ok, so where does this place this work on python-scipy for now? Is it a 100% no-go until *blas providers are reworked?

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

For me, yes, the first setp is to fix those blas providers.

Ok, but wouldn't it help having the python-scipy in here to have a good use-case for working with all this?

Yes, why not.

Ok, let's try to get a simple and clean python-scipy recipe in.

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines

There is libflame[1], a lapack implementation and Blis[2], a blas implementation.
libflame includes a compatibility layer, lapack2flame, which includes a complete LAPACK implementation.
While BLIS exports a new BLAS-like API, it also includes a BLAS compatibility layer which gives application developers access to BLIS implementations via traditional BLAS routine calls.

This have to be test but I guess it could replace lapack/clapack and blas implementation.
I suggest you to read the previous work on lapack/clapack and blas[3].

Quoted 5 lines
Quoted 3 lines

I am not at all a lapack user/developer, just trying to build something with it, so please bear with me.

No one really use lapack, python-scipy use it, by building python-scipy, you kind of become a lapack user.

True. But, but it does not make me any more knowledgeable of the API or internals.

Quoted 7 lines
Quoted 4 lines

You write that the python-scipy requires lapack (and thus implicitl that it does not work with clapack). Why is that? It seems to WORKSFORME, so what am I missing?

No, clapack is just a C implementation of lapack. It had been generated in C from fortran code with f2c.
Clapack is not maintained any more as lapack but there are still old.

I am with you so far.

Quoted 2 lines

Also, lapack need a fortran compiler as python-numpy/python-scipy then you should use lapack instead of clapack.

Well, I don't *have* to use lapack instead of clapack. I could, and you could argue I should.

But with this patch series, buildroot can build python-numpy and python-scipy using clapack, for a working numpy/scipy enabled rootfs. That sounds quite useful to me.

If I send a slightly modified version of this patch series, will you give it a review and see if we can agree on a form that can get merged now?

/Esben

#14 lore raw

PATCH v2 3/3 package/python-scipy: new package

Hi Esben,

Le lun. 23 sept. 2019 ? 14:46, Esben Haabendal <[email protected]> a ?crit :

Quoted 117 lines

Hi Alexandre

Quoted 1 line · Alexandre PAYEN

Le lun. 23 sept. 2019 ? 10:51, Esben Haabendal <[email protected]> a

?crit :

Quoted 10 lines
Quoted 9 lines
Quoted 3 lines · Alexandre PAYEN

This is exatcly what I did. I propose a virtual package for lapack/clapack. But after digging into lapack/clapack, we realise this is much harder than just adding a virtual package.

What is harder, adding python-scipy first?

No, lapack/clapack and blas.

Ok. So while delaying addition of python-scipy with a relatively simple virtual package change, we now have a much harder task at hand.

Quoted 5 lines
Quoted 5 lines

If so, I think you might been hindered by a cross-compilation unfriendly mechanism in numpy distutils, which I have got a fix into numpy for. Please take a look at

https://github.com/numpy/numpy/pull/14495/commits/153fc148eec60e5cbec0e80617f75a3a5dd2a3f8

Quoted 8 lines
Quoted 6 lines

and how I use that in my python-scipy recipe sent to this list.

With that, I think the python-scipy recipe looks pretty simple and innocent, and I don't think that merging it should be a problem for reworking lapack/clpack/*blas later on.

Yes, there was a cross-compilation problem. But we fix it poorly, I

assume

Quoted 1 line

your commit do the job.

I believe it does.

Quoted 2 lines

I think adding python-scipy is not relevant because we don't understand lapack/clapack and blas properly.

Well, addition of python-scipy is relevant for me, as I need to have support for it. It is a firm product requirement, so the lack of it requires me to carry it in an overlay or to maintain patches for it.

Quoted 1 line

We need to fix blas/lapack before adding packages which uses it.

It might be old, but clapack seems to do the job for python-scipy. So I don't see that we *need* to do something to blas/lapack before adding python-scipy, but it definitely sounds like it would be good to do something about it, with or without python-scipy.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

OpenBlas is using a bundled version of Lapack when gFortran

compiler

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

is available. Clapack is an old f2c'ed version of lapack, so it

can be

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

used without a Fortran Compiler. Since Python-scipy depends on Fortran, Clapack can be removed from python-scipy.

Actually there is an existing issue (discussed during the Buildroot hackathon) on *blas providers. We need to fix this before adding

new

Quoted 11 lines
Quoted 9 lines
Quoted 6 lines
Quoted 5 lines
Quoted 1 line

packages using *blas libraries :-/

Ok, so where does this place this work on python-scipy for now? Is it a 100% no-go until *blas providers are reworked?

For me, yes, the first setp is to fix those blas providers.

Ok, but wouldn't it help having the python-scipy in here to have a good use-case for working with all this?

Yes, why not.

Ok, let's try to get a simple and clean python-scipy recipe in.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

There is libflame[1], a lapack implementation and Blis[2], a blas implementation.
`libflame includes a compatibility layer, lapack2flame, which

includes a

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

complete LAPACK implementation.`
While BLIS exports a new BLAS-like API, it also includes a BLAS compatibility layer which gives application developers access to BLIS implementations via traditional BLAS routine calls.

This have to be test but I guess it could replace lapack/clapack and

blas

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

implementation.
I suggest you to read the previous work on lapack/clapack and blas[3].

I am not at all a lapack user/developer, just trying to build something with it, so please bear with me.

No one really use lapack, python-scipy use it, by building python-scipy, you kind of become a lapack user.

True. But, but it does not make me any more knowledgeable of the API or internals.

Quoted 5 lines
Quoted 4 lines

You write that the python-scipy requires lapack (and thus implicitl that it does not work with clapack). Why is that? It seems to WORKSFORME, so what am I missing?

No, clapack is just a C implementation of lapack. It had been generated

in

Quoted 2 lines

C from fortran code with f2c.
Clapack is not maintained any more as lapack but there are still old.

I am with you so far.

Quoted 2 lines

Also, lapack need a fortran compiler as python-numpy/python-scipy then you should use lapack instead of clapack.

Well, I don't *have* to use lapack instead of clapack. I could, and you could argue I should.

Quoted 8 lines

But with this patch series, buildroot can build python-numpy and python-scipy using clapack, for a working numpy/scipy enabled rootfs. That sounds quite useful to me.

If I send a slightly modified version of this patch series, will you give it a review and see if we can agree on a form that can get merged now?

Please, proceed, I will review it.

Quoted 3 lines

/Esben

Alexandre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190923/0586da5a/attachment.html>