Skip to contentSkip to conversation

python modules search in non english locale (zlib, ...)

PATCH New Sébastien Royen 3 participants 10 messages Mar 27, 2015 – Mar 31, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

setup.py: do not add crosscompile header locations if language is not english

With buildroot toolchain, gcc can be multi language.
Python package setup.py need gcc english output to work fine. (find zlib for example)
We force language to en_US for the need of output parsing.

setup.py 2015-03-27 23:44:48.482968892 +0100 renamed from setup.py 2015-03-27 23:55:53.738987211 +0100 +0 -1
Index: b/setup.py
===================================================================
@@ -414,7 +414,7 @@
         tmpfile = os.path.join(self.build_temp, 'gccpaths')
         if not os.path.exists(self.build_temp):
             os.makedirs(self.build_temp)
-        ret = os.system('%s -E -v - </dev/null 2>%s 1>/dev/null' %

(gcc, tmpfile))
+ ret = os.system('LANGUAGE=en_US %s -E -v - </dev/null 2>%s
1>/dev/null' % (gcc, tmpfile))
is_gcc = False
in_incdirs = False
inc_dirs = []

#1 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Dear S?bastien Royen,

Quoted 9 lines · S?bastien Royen

setup.py: do not add crosscompile header locations if language is not english

With buildroot toolchain, gcc can be multi language.
Python package setup.py need gcc english output to work fine. (find zlib for example)
We force language to en_US for the need of output parsing.

What you posted is not a Buildroot patch, so we cannot apply it. I guess it's a python or python3 patch. Can you rework this to make it actually usable by Buildroot, and also make sure whether the issue is applicable to python, python3, or both?

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#2 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Sorry,

I should not submit patch of patch too late in the evening. I hope the version below is better.
Not tested with python3, and not sure toolchain output language shouln't be fix in another way to avoid same problem on other packages.

Le 29/03/2015 14:46, Thomas Petazzoni a ?crit :

Quoted 12 lines

Dear S?bastien Royen,

Quoted 9 lines · S?bastien Royen

setup.py: do not add crosscompile header locations if language is not english

With buildroot toolchain, gcc can be multi language.
Python package setup.py need gcc english output to work fine. (find zlib for example)
We force language to en_US for the need of output parsing.

Quoted 8 lines

What you posted is not a Buildroot patch, so we cannot apply it. I guess it's a python or python3 patch. Can you rework this to make it actually usable by Buildroot, and also make sure whether the issue is applicable to python, python3, or both?

Thanks,

Thomas

--- a/package/python/116-enforce-cross-compile-headers-search.patch

 1970-01-01 01:00:00.000000000 +0100
+++ b/package/python/116-enforce-cross-compile-headers-search.patch  
 2015-03-27 23:58:21.478991279 +0100

@@ -0,0 +1,20 @@
+setup.py: do not add crosscompile header locations if language is not english
+
+With buildroot toolchain, gcc is multi language.
+setup.py need english output to work fine. (find zlib for example) +We force language to en_US for the need of output parsing. +
+Signed-off-by: Sebastien Royen <[email protected]> +Index: b/setup.py

+===================================================================
+--- a/setup.py    2015-03-27 23:55:53.738987211 +0100
++++ b/setup.py    2015-03-27 23:44:48.482968892 +0100

+@@ -414,7 +414,7 @@
+ tmpfile = os.path.join(self.build_temp, 'gccpaths')
+ if not os.path.exists(self.build_temp):
+ os.makedirs(self.build_temp)
+- ret = os.system('%s -E -v - </dev/null 2>%s 1>/dev/null' %
(gcc, tmpfile))
++ ret = os.system('LANGUAGE=en_US %s -E -v - </dev/null 2>%s
1>/dev/null' % (gcc, tmpfile))

+         is_gcc = False
+         in_incdirs = False
+         inc_dirs = []
#3 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Dear S?bastien Royen,

Quoted 2 lines · S?bastien Royen

I should not submit patch of patch too late in the evening. I hope the version below is better.

Yes and no. It's better because it's a Buildroot patch, but it's still not good since it's not a Git formatted patch.

Quoted 2 lines

Not tested with python3, and not sure toolchain output language shouln't be fix in another way to avoid same problem on other packages.

Maybe we should export LC_ALL globally, or something like that.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#4 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Quoted 7 lines · Thomas Petazzoni

Dear S?bastien Royen,

Quoted 2 lines · S?bastien Royen

I should not submit patch of patch too late in the evening. I hope the version below is better.

Quoted 7 lines

Yes and no. It's better because it's a Buildroot patch, but it's still not good since it's not a Git formatted patch.

Quoted 2 lines

Not tested with python3, and not sure toolchain output language shouln't be fix in another way to avoid same problem on other packages.

Maybe we should export LC_ALL globally, or something like that.

Can confirm, that I'm having the same issue with my German openSUSE 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL yet. Python 2.7.9.

The whole stuff got screwed only lately. I've been using python-dpkt, that requires PYTHON_ZLIB for ages and had no problems on this machine.

I think this bug (https://bugs.busybox.net/show_bug.cgi?id=7971) is also locale related.

Yegor

#5 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Dear Yegor Yefremov,

Quoted 3 lines · Yegor Yefremov

Can confirm, that I'm having the same issue with my German openSUSE 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL yet. Python 2.7.9.

Ok.

Quoted 3 lines

The whole stuff got screwed only lately. I've been using python-dpkt, that requires PYTHON_ZLIB for ages and had no problems on this machine.

Seems weird. Do we have an idea what broke things ? Can you try 2015.02, 2014.11 and see if the issue also happens (or not) ?

Quoted 2 lines

I think this bug (https://bugs.busybox.net/show_bug.cgi?id=7971) is also locale related.

Ok.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#6 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Quoted 8 lines · Thomas Petazzoni

Dear Yegor Yefremov,

Quoted 3 lines · Yegor Yefremov

Can confirm, that I'm having the same issue with my German openSUSE 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL yet. Python 2.7.9.

Quoted 8 lines

Ok.

Quoted 3 lines

The whole stuff got screwed only lately. I've been using python-dpkt, that requires PYTHON_ZLIB for ages and had no problems on this machine.

Seems weird. Do we have an idea what broke things ? Can you try 2015.02, 2014.11 and see if the issue also happens (or not) ?

Hm. Strange. 2014.11 seem to have the same issue. Can it be the change from Python 2.7.8 to 2.7.9?

# cat /etc/os-release
NAME=Buildroot
VERSION=2014.11
ID=buildroot
VERSION_ID=2014.11
PRETTY_NAME="Buildroot 2014.11"

I'm also working with en englich Xubuntu VM. I can imagine, that I haven't made updates/cleaning on my suse machine. Most upstream work was made with Xubuntu.

Quoted 10 lines
Quoted 2 lines

I think this bug (https://bugs.busybox.net/show_bug.cgi?id=7971) is also locale related.

Ok.

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

#7 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Quoted 29 lines · Yegor Yefremov
Quoted 8 lines · Thomas Petazzoni

Dear Yegor Yefremov,

Quoted 3 lines · Yegor Yefremov

Can confirm, that I'm having the same issue with my German openSUSE 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL yet. Python 2.7.9.

Quoted 8 lines

Ok.

Quoted 3 lines

The whole stuff got screwed only lately. I've been using python-dpkt, that requires PYTHON_ZLIB for ages and had no problems on this machine.

Seems weird. Do we have an idea what broke things ? Can you try 2015.02, 2014.11 and see if the issue also happens (or not) ?

Hm. Strange. 2014.11 seem to have the same issue. Can it be the change from Python 2.7.8 to 2.7.9?

# cat /etc/os-release
NAME=Buildroot
VERSION=2014.11
ID=buildroot
VERSION_ID=2014.11
PRETTY_NAME="Buildroot 2014.11"
Quoted 3 lines

I'm also working with en englich Xubuntu VM. I can imagine, that I haven't made updates/cleaning on my suse machine. Most upstream work was made with Xubuntu.

Python 3.4.2 seems to have this issue too.

Yegor

#8 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Quoted 31 lines · Yegor Yefremov
Quoted 29 lines · Yegor Yefremov
Quoted 8 lines · Thomas Petazzoni

Dear Yegor Yefremov,

Quoted 3 lines · Yegor Yefremov

Can confirm, that I'm having the same issue with my German openSUSE 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL yet. Python 2.7.9.

Quoted 8 lines

Ok.

Quoted 3 lines

The whole stuff got screwed only lately. I've been using python-dpkt, that requires PYTHON_ZLIB for ages and had no problems on this machine.

Seems weird. Do we have an idea what broke things ? Can you try 2015.02, 2014.11 and see if the issue also happens (or not) ?

Hm. Strange. 2014.11 seem to have the same issue. Can it be the change from Python 2.7.8 to 2.7.9?

# cat /etc/os-release
NAME=Buildroot
VERSION=2014.11
ID=buildroot
VERSION_ID=2014.11
PRETTY_NAME="Buildroot 2014.11"
Quoted 5 lines
Quoted 3 lines

I'm also working with en englich Xubuntu VM. I can imagine, that I haven't made updates/cleaning on my suse machine. Most upstream work was made with Xubuntu.

Python 3.4.2 seems to have this issue too.

Tried to build image via Sourcery CodeBench ARM 2014.05. Python 2.7.9 shows no problems. But when I'm doing the same with BR's toolchain, zlib module won't be found at runtime.

BR2_arm=y
BR2_cortex_a8=y
BR2_CCACHE=y
BR2_TOOLCHAIN_BUILDROOT_LARGEFILE=y
BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
BR2_TOOLCHAIN_BUILDROOT_INET_RPC=y
BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_BINUTILS_VERSION_2_25=y
BR2_GCC_VERSION_4_9_X=y
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_ZLIB=y

#9 lore raw

PATCH 1/1 python modules search in non english locale (zlib, ...)

Le 31/03/2015 11:05, Yegor Yefremov a ?crit :

Quoted 38 lines
Quoted 28 lines · Yegor Yefremov
Quoted 26 lines · Yegor Yefremov
Quoted 7 lines · Thomas Petazzoni

Dear Yegor Yefremov,

Quoted 3 lines · Yegor Yefremov

Can confirm, that I'm having the same issue with my German openSUSE 12.1 machine. Exporting LANGUAGE=en_US helped. haven't tried LC_ALL yet. Python 2.7.9.

Quoted 7 lines

Ok.

Quoted 3 lines

The whole stuff got screwed only lately. I've been using python-dpkt, that requires PYTHON_ZLIB for ages and had no problems on this machine.

Seems weird. Do we have an idea what broke things ? Can you try 2015.02, 2014.11 and see if the issue also happens (or not) ?

Hm. Strange. 2014.11 seem to have the same issue. Can it be the change from Python 2.7.8 to 2.7.9?

# cat /etc/os-release
NAME=Buildroot
VERSION=2014.11
ID=buildroot
VERSION_ID=2014.11
PRETTY_NAME="Buildroot 2014.11"
Quoted 4 lines
Quoted 3 lines

I'm also working with en englich Xubuntu VM. I can imagine, that I haven't made updates/cleaning on my suse machine. Most upstream work was made with Xubuntu.

Python 3.4.2 seems to have this issue too.

Tried to build image via Sourcery CodeBench ARM 2014.05. Python 2.7.9 shows no problems. But when I'm doing the same with BR's toolchain, zlib module won't be found at runtime.

Quoted 12 lines

BR2_arm=y
BR2_cortex_a8=y
BR2_CCACHE=y
BR2_TOOLCHAIN_BUILDROOT_LARGEFILE=y
BR2_TOOLCHAIN_BUILDROOT_INET_IPV6=y
BR2_TOOLCHAIN_BUILDROOT_INET_RPC=y
BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_BINUTILS_VERSION_2_25=y
BR2_GCC_VERSION_4_9_X=y
BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_ZLIB=y

Hi,

Problem is on parsing output of : gcc -E -v - </dev/null 1>/dev/null
Maybe it would be better to fix setup.py in order to be generic.
Output with en_US is like :
...
#include <...> search starts here:
/my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include-fixed /my_path/bsp/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/../../../../arm-buildroot-linux-gnueabihf/include /my_path/buildroot/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot/usr/include
End of search list.
...

Output with de is like :
...
#include <...> - Suche beginnt hier:
/my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include-fixed /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/../../../../arm-buildroot-linux-gnueabihf/include /my_path/buildroot/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot/usr/include
Ende der Suchliste.
...

Output with fr version is :
la recherche pour #include <...> d?bute ici:
/my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/include-fixed /my_path/buildroot/output/host/usr/lib/gcc/arm-buildroot-linux-gnueabihf/4.8.4/../../../../arm-buildroot-linux-gnueabihf/include /my_path/buildroot/output/host/usr/arm-buildroot-linux-gnueabihf/sysroot/usr/include
Fin de la liste de recherche.
...

Or maybe we should keep only C language in toolchain build ? ( something to do with BR2_ENABLE_LOCALE_WHITELIST ? )

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20150331/1efe615f/attachment.html>