package/python-pillow: new package

PATCH Superseded Angelo Compagnucci 3 participants 7 messages Feb 20, 2016 – Feb 24, 2016

lore

Share & tools

lore

Angelo Compagnucci

author

2016-02-20 09:55 UTC #0 lore raw

PATCH v4 1/1 package/python-pillow: new package

This patch adds python-pillow, the friendly python image library fork.

Signed-off-by Angelo Compagnucci <[email protected]>

---
Changes:
v3 -> v4:

* Fixing a couple of typos

v2 -> v3:

* Simplified package
* Better handling of setup type
* Adding hash file

v1 -> v2:

* Added patch to remove platform guessing, now headers should be searched in buildroot folders
* Fixed optional configure options

 package/Config.in                                  |  1 +
 ...setup.py-removing-unneeded-platform-guess.patch | 85 ++++++++++++++++++++++
 package/python-pillow/Config.in                    | 12 +++
 package/python-pillow/python-pillow.hash           |  2 +
 package/python-pillow/python-pillow.mk             | 17 +++++
 5 files changed, 117 insertions(+)
 create mode 100644 package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch
 create mode 100644 package/python-pillow/Config.in
 create mode 100644 package/python-pillow/python-pillow.hash
 create mode 100644 package/python-pillow/python-pillow.mk

package/Config.in +1 -0

index 3eb3618..55f16cc 100644
@@ -674,6 +674,7 @@ menu "External python modules"
 	source "package/python-paho-mqtt/Config.in"
 	source "package/python-pam/Config.in"
 	source "package/python-paramiko/Config.in"
+	source "package/python-pillow/Config.in"
 	source "package/python-posix-ipc/Config.in"
 	source "package/python-protobuf/Config.in"
 	source "package/python-psutil/Config.in"

package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch new file +85 -0

85 lines not on this page · raw message

package/python-pillow/Config.in new file +12 -0

new file mode 100644
index 0000000..d5c3809
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_PYTHON_PILLOW
+	bool "python-pillow"
+	help
+	  Pillow is the "friendly" PIL fork by Alex Clark and Contributors. PIL is
+	  the Python Imaging Library by Fredrik Lundh and Contributors.
+
+	  Pillow relies on external libraries to provide support various
+	  image formats. Select the corresponding package(s) to get this
+	  support. Pillow can use jpeg, zlib (for PNG), tiff, freetype, webp,
+	  and openjpeg (JPEG-2000).
+
+	  https://pypi.python.org/pypi/Pillow/

package/python-pillow/python-pillow.hash new file +2 -0

new file mode 100644
index 0000000..fd2978a
@@ -0,0 +1,2 @@
+# sha256 locally computed
+sha256 780f21465e2b7690fc55925188373cd54668ea4d71964f971e1fea4bc16d365e  python-pillow-3.0.0.tar.gz

package/python-pillow/python-pillow.mk new file +17 -0

new file mode 100644
index 0000000..a956a2e
@@ -0,0 +1,17 @@
+################################################################################
+#
+# python-pillow
+#
+################################################################################
+
+PYTHON_PILLOW_VERSION = 3.0.0
+PYTHON_PILLOW_SITE = $(call github,python-pillow,Pillow,$(PYTHON_PILLOW_VERSION))
+PYTHON_PILLOW_SETUP_TYPE = setuptools
+PYTHON_PILLOW_DEPENDENCIES = $(if $(BR2_PACKAGE_JPEG),jpeg) \
+      $(if $(BR2_PACKAGE_ZLIB),zlib) \
+      $(if $(BR2_PACKAGE_TIFF),tiff) \
+      $(if $(BR2_PACKAGE_FREETYPE),freetype) \
+      $(if $(BR2_PACKAGE_WEBP),webp) \
+      $(if $(BR2_PACKAGE_OPENJPEG),openjpeg)
+
+$(eval $(python-package))

Signature · 1 line

1.9.1

Thomas Petazzoni

2016-02-21 14:26 UTC #1 lore raw

PATCH v4 1/1 package/python-pillow: new package

Angelo,

Quoted 3 lines · Angelo Compagnucci

This patch adds python-pillow, the friendly python image library fork.
Signed-off-by Angelo Compagnucci <[email protected]>

This package does not build when jpeg is disabled:

running build_ext
Traceback (most recent call last):
File "setup.py", line 711, in <module>
zip_safe=not debug_build(),
File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/command/build.py", line 126, in run
    self.run_command(cmd_name)
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/command/build_ext.py", line 342, in run
    self.build_extensions()
  File "setup.py", line 459, in build_extensions
    % (f, f))

ValueError: --enable-jpeg requested but jpeg not found, aborting.

It seems like you should pass explicit --enable/--disable options, since at least jpeg and zlib are by default expected to be enabled. See https://pillow.readthedocs.org/en/latest/installation.html#external-libraries.

Unfortunately, those options are only recognized when calling "setup.py build_ext", not when calling "setup.py build" as the python-package infrastructure is doing. So you might need to override the build commands for this package.

Quoted 6 lines

package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch new file +0 -0
new file mode 100644
index 0000000..5f43013

package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch line 9

Quoted 9 lines

package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch new file +9 -0

@@ -0,0 +1,85 @@
+From cb8c67c0b7ee805100c381300ea29262e8b5838a Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <[email protected]>
+Date: Thu, 22 Oct 2015 22:45:31 +0200
+Subject: [PATCH] setup.py: removing unneeded platform guess
+
+Platform guess is not needed when cross compiling on buildroot
+so removing it.
+
+Signed-off-by: Angelo Compagnucci <[email protected]>

Can you submit this patch upstream or at least report the issue?

Thanks!

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Angelo Compagnucci

author

2016-02-23 22:18 UTC #2 lore raw

PATCH v4 1/1 package/python-pillow: new package

Dear Thomas Petazzoni,

2016-02-21 15:26 GMT+01:00 Thomas Petazzoni
<[email protected]>:

Quoted 31 lines

Angelo,
Quoted 3 lines · Angelo Compagnucci
This patch adds python-pillow, the friendly python image library fork.
Signed-off-by Angelo Compagnucci <[email protected]>
This package does not build when jpeg is disabled:
running build_ext
Traceback (most recent call last):
File "setup.py", line 711, in <module>
zip_safe=not debug_build(),
File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/command/build.py", line 126, in run
    self.run_command(cmd_name)
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/thomas/projets/buildroot/output/host/usr/lib/python3.4/distutils/command/build_ext.py", line 342, in run
    self.build_extensions()
  File "setup.py", line 459, in build_extensions
    % (f, f))
ValueError: --enable-jpeg requested but jpeg not found, aborting.

Quoted 8 lines

It seems like you should pass explicit --enable/--disable options, since at least jpeg and zlib are by default expected to be enabled. See https://pillow.readthedocs.org/en/latest/installation.html#external-libraries.
Unfortunately, those options are only recognized when calling "setup.py build_ext", not when calling "setup.py build" as the python-package infrastructure is doing. So you might need to override the build commands for this package.

Right.

I actually implemented something lie this in pkg-python.mk

[...]
$(2)_BUILD_TARGET ?= build
[...]
$(2)_BASE_BUILD_TGT = $$($(2)_BUILD_TARGET)
[...]

and seems to work and not breaking other packages. This way I can add:

PYTHON_PILLOW_BUILD_TARGET = build_ext

What do you think?

Quoted 9 lines

Quoted 9 lines
package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch new file +3 -0
new file mode 100644
index 0000000..5f43013
@@ -0,0 +1,85 @@
+From cb8c67c0b7ee805100c381300ea29262e8b5838a Mon Sep 17 00:00:00 2001
+From: Angelo Compagnucci <[email protected]>
+Date: Thu, 22 Oct 2015 22:45:31 +0200

Quoted 8 lines

package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch line 9
Quoted 6 lines
package/python-pillow/0001-setup.py-removing-unneeded-platform-guess.patch new file +6 -0
+Subject: [PATCH] setup.py: removing unneeded platform guess
+
+Platform guess is not needed when cross compiling on buildroot
+so removing it.
+
+Signed-off-by: Angelo Compagnucci <[email protected]>
Can you submit this patch upstream or at least report the issue?

I don't think so. That piece of code is used to guess the platform on which your are compiling with the assumption that host arch == target arch. Probably, pillow is not originally designed to be cross compiled and the build system would require a rewrite for a more structured approach.
The only way to let buildroot passing the appropriate options to build and setup is to remove that piece of code.

Quoted 8 lines

Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Signature · 1 line

Profile: http://it.linkedin.com/in/compagnucciangelo

Thomas Petazzoni

2016-02-23 22:35 UTC #3 lore raw

PATCH v4 1/1 package/python-pillow: new package

Hello,

Quoted 10 lines · Angelo Compagnucci

Right.
I actually implemented something lie this in pkg-python.mk
[...]
$(2)_BUILD_TARGET ?= build
[...]
$(2)_BASE_BUILD_TGT = $$($(2)_BUILD_TARGET)
[...]

Quoted 6 lines

and seems to work and not breaking other packages. This way I can add:
PYTHON_PILLOW_BUILD_TARGET = build_ext
What do you think?

Since it's the first package to require this, I would rather suggest to override PYTHON_PILLOW_BUILD_CMDS in python-pillow.mk, so that this hack is limited to this package.

Should more packages need this in the future, we can add better support in the infrastructure.

But a Python package that needs to pass options only to build_ext is IMO broken, and should allow them to be passed to the "build" target. build_ext is more or less an internal step, which we shouldn't have to worry about.

Quoted 11 lines

Quoted 3 lines
Quoted 1 line
+Signed-off-by: Angelo Compagnucci <[email protected]>
Can you submit this patch upstream or at least report the issue?
I don't think so. That piece of code is used to guess the platform on which your are compiling with the assumption that host arch == target arch. Probably, pillow is not originally designed to be cross compiled and the build system would require a rewrite for a more structured approach.
The only way to let buildroot passing the appropriate options to build and setup is to remove that piece of code.

Well, you can make it upstreamable by making it understand some environment variable, or better some option, to indicate that we are cross-compiling.

So yes, your patch is not acceptable upstream as is, but my comment is precisely that is should ideally be in a form that can potentially be accepted upstream.

Thanks!

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Arnout Vandecappelle

2016-02-23 22:45 UTC #4 lore raw

PATCH v4 1/1 package/python-pillow: new package

Quoted 17 lines · Thomas Petazzoni

Quoted 15 lines · Angelo Compagnucci
Quoted 15 lines
Right.
I actually implemented something lie this in pkg-python.mk
[...]
$(2)_BUILD_TARGET ?= build
[...]
$(2)_BASE_BUILD_TGT = $$($(2)_BUILD_TARGET)
[...]
and seems to work and not breaking other packages. This way I can add:
PYTHON_PILLOW_BUILD_TARGET = build_ext

Quoted 7 lines

Quoted 1 line
Quoted 1 line
What do you think?
Since it's the first package to require this, I would rather suggest to override PYTHON_PILLOW_BUILD_CMDS in python-pillow.mk, so that this hack is limited to this package.
Should more packages need this in the future, we can add better support in the infrastructure.

Hm, good point as well. Especially because the build commands are just a few lines.

I notice now that in pkg-python we define _BASE_BUILD_TGT in four different
places, and it's always 'build'. Probably can be removed...

Regards,
Arnout

Signature · 6 lines

Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

Angelo Compagnucci

author

2016-02-24 07:06 UTC #5 lore raw

PATCH v4 1/1 package/python-pillow: new package

In reply to Thomas Petazzoni

Dear Thomas Petazzoni,

2016-02-23 23:35 GMT+01:00 Thomas Petazzoni
<[email protected]>:

Quoted 19 lines

Hello,
Quoted 15 lines · Angelo Compagnucci
Right.
I actually implemented something lie this in pkg-python.mk
[...]
$(2)_BUILD_TARGET ?= build
[...]
$(2)_BASE_BUILD_TGT = $$($(2)_BUILD_TARGET)
[...]
and seems to work and not breaking other packages. This way I can add:
PYTHON_PILLOW_BUILD_TARGET = build_ext

Quoted 8 lines

Quoted 1 line
What do you think?
Since it's the first package to require this, I would rather suggest to override PYTHON_PILLOW_BUILD_CMDS in python-pillow.mk, so that this hack is limited to this package.
Should more packages need this in the future, we can add better support in the infrastructure.

I just sent a patch in which I implemented custom BUILD_CMDS and INSTALL_TARGET_CMDS.

Quoted 18 lines

But a Python package that needs to pass options only to build_ext is IMO broken, and should allow them to be passed to the "build" target. build_ext is more or less an internal step, which we shouldn't have to worry about.
Quoted 11 lines
Quoted 3 lines
Quoted 1 line
+Signed-off-by: Angelo Compagnucci <[email protected]>
Can you submit this patch upstream or at least report the issue?
I don't think so. That piece of code is used to guess the platform on which your are compiling with the assumption that host arch == target arch. Probably, pillow is not originally designed to be cross compiled and the build system would require a rewrite for a more structured approach.
The only way to let buildroot passing the appropriate options to build and setup is to remove that piece of code.

Quoted 7 lines

Well, you can make it upstreamable by making it understand some environment variable, or better some option, to indicate that we are cross-compiling.
So yes, your patch is not acceptable upstream as is, but my comment is precisely that is should ideally be in a form that can potentially be accepted upstream.

IMHO not having pillow in Buildroot in really a pity. I used it so many times in various projects that I was shocked when I realized it was lacking.

I would invest some time to try to understand better the build system and try to submit an upstream patch for a fully cross compilation, in the meantime, could you accept this patch?

It now looks good to me!

Quoted 8 lines

Thanks!
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Signature · 1 line

Profile: http://it.linkedin.com/in/compagnucciangelo

Arnout Vandecappelle

2016-02-23 22:36 UTC #6 lore raw

PATCH v4 1/1 package/python-pillow: new package

In reply to Angelo Compagnucci

Quoted 4 lines · Angelo Compagnucci

Dear Thomas Petazzoni,
2016-02-21 15:26 GMT+01:00 Thomas Petazzoni
<[email protected]>:

[snip]

Quoted 19 lines

Quoted 8 lines
It seems like you should pass explicit --enable/--disable options, since at least jpeg and zlib are by default expected to be enabled. See https://pillow.readthedocs.org/en/latest/installation.html#external-libraries.
Unfortunately, those options are only recognized when calling "setup.py build_ext", not when calling "setup.py build" as the python-package infrastructure is doing. So you might need to override the build commands for this package.
Right.
I actually implemented something lie this in pkg-python.mk
[...]
$(2)_BUILD_TARGET ?= build
[...]
$(2)_BASE_BUILD_TGT = $$($(2)_BUILD_TARGET)
[...]

Quoted 6 lines

and seems to work and not breaking other packages. This way I can add:
PYTHON_PILLOW_BUILD_TARGET = build_ext
What do you think?

Looks like a good idea to me - parallel with autotools-package.

Regards,
Arnout

[snip]

Signature · 6 lines

Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF