uclibc: deprecate locale support

PATCH New Waldemar Brodkorb 3 participants 7 messages Jun 28, 2015 – Jun 29, 2015

Share & tools

Waldemar Brodkorb

author

2015-06-28 07:14 UTC #0 lore raw

PATCH 1/1 uclibc: deprecate locale support

The quality of the locale implementation in uClibc seems to be bad. Recently a discussion on the uClibc mailinglist showed that the uClibc maintainer has no time to fix remaining issues in the locale implementation.
http://lists.uclibc.org/pipermail/uclibc/2015-June/049000.html

Because of the bad state of the code I would like to deprecate it. I never used it myself and see no good reason why I would need it for an embedded system.

I believe not many people are using it. For good working locale support there is still GNU libc and musl.

Signed-off-by Waldemar Brodkorb <[email protected]>

---
 Config.in.legacy         |    6 ++++++
 package/uclibc/Config.in |    8 --------
 2 files changed, 6 insertions(+), 8 deletions(-)

Config.in.legacy +6 -0

index d86d585..1814acb 100644
@@ -107,6 +107,12 @@ endif
 ###############################################################################
 comment "Legacy options removed in 2015.08"
 
+config BR2_TOOLCHAIN_BUILDROOT_LOCALE
+	bool "Toolchain locale/i18n setting removed"
+	select BR2_LEGACY
+	help
+	  uClibc locale support have been removed.
+
 config BR2_TARGET_UBOOT_NETWORK
 	bool "U-Boot custom network settings removed"
 	select BR2_LEGACY

package/uclibc/Config.in +0 -8

index 15a590f..03d7171 100644
@@ -100,14 +100,6 @@ config BR2_TOOLCHAIN_BUILDROOT_WCHAR
 	  wide characters (i.e characters longer than 8 bits, needed
 	  for locale support).
 
-config BR2_TOOLCHAIN_BUILDROOT_LOCALE
-	bool "Enable toolchain locale/i18n support"
-	select BR2_TOOLCHAIN_BUILDROOT_WCHAR
-	select BR2_ENABLE_LOCALE
-	help
-	  Enable this option if you want your toolchain to support
-	  localization and internationalization.
-
 choice
 	prompt "Thread library implementation"
 	default BR2_PTHREADS_NATIVE if BR2_UCLIBC_VERSION_SUPPORTS_NPTL

Signature · 1 line

1.7.10.4

Yann E. MORIN

2015-06-28 09:50 UTC #1 lore raw

PATCH 1/1 uclibc: deprecate locale support

Waldemar, All,

On 2015-06-28 09:14 +0200, Waldemar Brodkorb spake thusly:

Quoted 6 lines

The quality of the locale implementation in uClibc seems to be bad. Recently a discussion on the uClibc mailinglist showed that the uClibc maintainer has no time to fix remaining issues in the locale implementation.
http://lists.uclibc.org/pipermail/uclibc/2015-June/049000.html

Quoted 8 lines

Because of the bad state of the code I would like to deprecate it. I never used it myself and see no good reason why I would need it for an embedded system.
I believe not many people are using it. For good working locale support there is still GNU libc and musl.
Signed-off-by Waldemar Brodkorb <[email protected]>

While on principle I'm not opposed to deprecating locale support in uClibc altogether.

However, we've had locales working in uClinc so far, it's only with uClibc-ng that we started to have build failures (AFAICS).

So, I find it a bit cavalier that we remove support for locales when uClibc-ng / uClibc is used, like we'd hide the dirt under the rug.

Also, when we add something to the legacy entries, this is not deprecating, this is removing. Deprecating is done by adding a dependency to a symbol, like is done now for the blackbox package.

But for what it's worth:

Reviewed-by "Yann E. MORIN" <[email protected]>

Regards,
Yann E. MORIN.

Quoted 48 lines

---
 Config.in.legacy         |    6 ++++++
 package/uclibc/Config.in |    8 --------
 2 files changed, 6 insertions(+), 8 deletions(-)

Config.in.legacy +6 -0

index d86d585..1814acb 100644
@@ -107,6 +107,12 @@ endif
 ###############################################################################
 comment "Legacy options removed in 2015.08"
 
+config BR2_TOOLCHAIN_BUILDROOT_LOCALE
+	bool "Toolchain locale/i18n setting removed"
+	select BR2_LEGACY
+	help
+	  uClibc locale support have been removed.
+
 config BR2_TARGET_UBOOT_NETWORK
 	bool "U-Boot custom network settings removed"
 	select BR2_LEGACY

package/uclibc/Config.in +0 -8

index 15a590f..03d7171 100644
@@ -100,14 +100,6 @@ config BR2_TOOLCHAIN_BUILDROOT_WCHAR
 	  wide characters (i.e characters longer than 8 bits, needed
 	  for locale support).
 
-config BR2_TOOLCHAIN_BUILDROOT_LOCALE
-	bool "Enable toolchain locale/i18n support"
-	select BR2_TOOLCHAIN_BUILDROOT_WCHAR
-	select BR2_ENABLE_LOCALE
-	help
-	  Enable this option if you want your toolchain to support
-	  localization and internationalization.
-
 choice
 	prompt "Thread library implementation"
 	default BR2_PTHREADS_NATIVE if BR2_UCLIBC_VERSION_SUPPORTS_NPTL

Signature · 6 lines

1.7.10.4

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 6 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Waldemar Brodkorb

author

2015-06-28 10:02 UTC #2 lore raw

PATCH 1/1 uclibc: deprecate locale support

Hi,
Yann E. MORIN wrote,

Quoted 16 lines

Waldemar, All,
On 2015-06-28 09:14 +0200, Waldemar Brodkorb spake thusly:
Quoted 13 lines
The quality of the locale implementation in uClibc seems to be bad. Recently a discussion on the uClibc mailinglist showed that the uClibc maintainer has no time to fix remaining issues in the locale implementation.
http://lists.uclibc.org/pipermail/uclibc/2015-June/049000.html
Because of the bad state of the code I would like to deprecate it. I never used it myself and see no good reason why I would need it for an embedded system.
I believe not many people are using it. For good working locale support there is still GNU libc and musl.

Quoted 7 lines

Quoted 1 line
Signed-off-by Waldemar Brodkorb <[email protected]>
While on principle I'm not opposed to deprecating locale support in uClibc altogether.
However, we've had locales working in uClinc so far, it's only with uClibc-ng that we started to have build failures (AFAICS).

I am really sure this also happens with uClibc git master. It worked for 0.9.33.2 in some way, but it is broken in latest code. So uclibc-ng inherited that.

Quoted 2 lines

So, I find it a bit cavalier that we remove support for locales when uClibc-ng / uClibc is used, like we'd hide the dirt under the rug.

I planned to remove/deprecate support for locales before the autobuilders started to show the bad situation. Just didn't had the time to prepare a patch.

May be it is just time to get rid of this unmaintained functionality.

If someone have a usecase for locales and prepares a patch, I will not reject it.

best regards
Waldemar

Thomas Petazzoni

2015-06-28 12:01 UTC #3 lore raw

PATCH 1/1 uclibc: deprecate locale support

In reply to Waldemar Brodkorb

Dear Waldemar Brodkorb,

Quoted 6 lines · Waldemar Brodkorb

The quality of the locale implementation in uClibc seems to be bad. Recently a discussion on the uClibc mailinglist showed that the uClibc maintainer has no time to fix remaining issues in the locale implementation.
http://lists.uclibc.org/pipermail/uclibc/2015-June/049000.html

Quoted 8 lines

Because of the bad state of the code I would like to deprecate it. I never used it myself and see no good reason why I would need it for an embedded system.
I believe not many people are using it. For good working locale support there is still GNU libc and musl.
Signed-off-by Waldemar Brodkorb <[email protected]>

Can you assess the impact on the packages we currently have in Buildroot ? I.e, which packages can only work with locale support?

If we remove BR2_TOOLCHAIN_BUILDROOT_LOCALE, then there's a lot of possible further cleanup: BR2_ENABLE_LOCALE should be removed, all packages doing select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE should be changed, etc.

Also, do you have an idea of what is the state of the locale support in musl ? Does musl provide the functions provided by libiconv ?

It would be good to take this opportunity to make sure the locale support is OK.

Thanks,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Waldemar Brodkorb

author

2015-06-28 12:58 UTC #4 lore raw

PATCH 1/1 uclibc: deprecate locale support

Hi Thomas,
Thomas Petazzoni wrote,

Quoted 13 lines

Dear Waldemar Brodkorb,
Quoted 10 lines · Waldemar Brodkorb
The quality of the locale implementation in uClibc seems to be bad. Recently a discussion on the uClibc mailinglist showed that the uClibc maintainer has no time to fix remaining issues in the locale implementation.
http://lists.uclibc.org/pipermail/uclibc/2015-June/049000.html
Because of the bad state of the code I would like to deprecate it. I never used it myself and see no good reason why I would need it for an embedded system.

Quoted 7 lines

Quoted 4 lines
I believe not many people are using it. For good working locale support there is still GNU libc and musl.
Signed-off-by Waldemar Brodkorb <[email protected]>
Can you assess the impact on the packages we currently have in Buildroot ? I.e, which packages can only work with locale support?

I think no package really needs locale support in the C library.

May be there is some misunderstanding about the locale stuff. Here is how I understand the stuff.

I think we have mixed up two different things here:
- iconv: perform character set conversion, needed for a lot of packages, f.e. Kodi. Is included in Glibc and Musl. Libiconv can be used with uClibc-ng/uClibc.
- locale: defines the user's language, country, ..
If you want f.e. german error messages on your embedded system you would need locale support and locale data.
env LC_ALL=de_DE ls foo

Who is really using the locale stuff?
I think the iconv functions, which are automatically enabled when UCLIBC_HAS_LOCALE is set are neither complete not used externally. If libiconv is to big, libiconv-tiny for simple UTF-8 <-> ASCII conversion could be used.

Quoted 4 lines

If we remove BR2_TOOLCHAIN_BUILDROOT_LOCALE, then there's a lot of possible further cleanup: BR2_ENABLE_LOCALE should be removed, all packages doing select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE should be changed, etc.

May be something like this:
select BR2_PACKAGE_LIBICONV if BR2_LIBC_WITHOUT_ICONV

Quoted 2 lines

Also, do you have an idea of what is the state of the locale support in musl ? Does musl provide the functions provided by libiconv ?

Yes. Like GNU libc, musl already provides iconv functions out of the box.

Quoted 2 lines

It would be good to take this opportunity to make sure the locale support is OK.

Yes, that would be good.

best regards
Waldemar

Thomas Petazzoni

2015-06-28 13:08 UTC #5 lore raw

PATCH 1/1 uclibc: deprecate locale support

Hello,

Quoted 5 lines · Waldemar Brodkorb

I think no package really needs locale support in the C library.
May be there is some misunderstanding about the locale stuff. Here is how I understand the stuff.

Quoted 8 lines

I think we have mixed up two different things here:
- iconv: perform character set conversion, needed for a lot of packages, f.e. Kodi. Is included in Glibc and Musl. Libiconv can be used with uClibc-ng/uClibc.
- locale: defines the user's language, country, ..
If you want f.e. german error messages on your embedded system you would need locale support and locale data.
env LC_ALL=de_DE ls foo

Fully agreed here. But in uClibc, enabling locale support actually gives both: the iconv functions *and* the locale support itself.

Quoted 4 lines

Who is really using the locale stuff?
I think the iconv functions, which are automatically enabled when UCLIBC_HAS_LOCALE is set are neither complete not used externally.

Of course they are used externally: many of our packages do "select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE" because they unconditionally use the iconv function, and uClibc only provide these functions if locale support is enabled. From libc/misc/wchar/Makefile.in (in original uClibc, I haven't looked at uClibc-ng) :

CSRC-$(UCLIBC_HAS_LOCALE) += iconv.c

Quoted 2 lines

If libiconv is to big, libiconv-tiny for simple UTF-8 <-> ASCII conversion could be used.

This could be useful, but this would be a separate matter.

Quoted 7 lines

Quoted 4 lines
If we remove BR2_TOOLCHAIN_BUILDROOT_LOCALE, then there's a lot of possible further cleanup: BR2_ENABLE_LOCALE should be removed, all packages doing select BR2_PACKAGE_LIBICONV if !BR2_ENABLE_LOCALE should be changed, etc.
May be something like this:
select BR2_PACKAGE_LIBICONV if BR2_LIBC_WITHOUT_ICONV

Yes, correct.

Quoted 6 lines

Quoted 3 lines
Also, do you have an idea of what is the state of the locale support in musl ? Does musl provide the functions provided by libiconv ?
Yes. Like GNU libc, musl already provides iconv functions out of the box.

Ok, so BR2_LIBC_WITHOUT_ICONV would be simple: it's y if uClibc, and n otherwise.

Or should we instead have in uClibc the iconv functions enabled by default (by having a UCLIBC_HAS_ICONV option separate from UCLIBC_HAS_LOCALE), so that uClibc is like all other C libraries from an iconv() point of view ? And of course, get rid completely of locale support in uClibc ?

There are still lots of packages that use BR2_ENABLE_LOCALE in Buildroot, we will have to check what they do exactly with it.

Best regards,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Waldemar Brodkorb

author

2015-06-29 17:47 UTC #6 lore raw

PATCH 1/1 uclibc: deprecate locale support

Hi,
Thomas Petazzoni wrote,

Quoted 18 lines

Hello,
Quoted 5 lines · Waldemar Brodkorb
I think no package really needs locale support in the C library.
May be there is some misunderstanding about the locale stuff. Here is how I understand the stuff.
Quoted 8 lines
I think we have mixed up two different things here:
- iconv: perform character set conversion, needed for a lot of packages, f.e. Kodi. Is included in Glibc and Musl. Libiconv can be used with uClibc-ng/uClibc.
- locale: defines the user's language, country, ..
If you want f.e. german error messages on your embedded system you would need locale support and locale data.
env LC_ALL=de_DE ls foo

Quoted 2 lines

Fully agreed here. But in uClibc, enabling locale support actually gives both: the iconv functions *and* the locale support itself.

Yeah, unfortunately both is coupled together as another research has proofed.

So the best option for 1.0.x to fix the build issue.
I tracked it down to a parallel build issue. When number of jobs is set to 1, the error does not happen.
I thought using the make target pregen instead of header would be a good workaround, but the problem still exist in Xtensa builds. The ARM builds work after this change.

Any parallel make gurus available?
It only happens hen LOCALE support is activated.

best regards
Waldemar