Skip to contentSkip to conversation

Generation of locales: made call to tr more robust

PATCH New Thierry Bultel 3 participants 8 messages Nov 8, 2013 – Nov 10, 2013 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

From: tbultel <[email protected]>

When calling 'tr' without quoting braces, bash can make really weird things if there are existing 'single-letter-named' directories
eg:
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
aaa
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
AAA

The (quick) analysis is that the callee (tr) argvs then contain 'm' thus the translation does not work

Using quotes works around it
tbultel at laois:~/test$ echo AAA | tr '[A-Z]' '[a-z]' aaa

---
 Makefile |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +1 -1
index 1496bd7..7f835c0 100644
@@ -548,7 +548,7 @@ target-generatelocales: host-localedef
 		I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
 		$(HOST_DIR)/usr/bin/localedef \
 			--prefix=$(TARGET_DIR) \
-			--`echo $(BR2_ENDIAN) | tr [A-Z] [a-z]`-endian \
+			--`echo $(BR2_ENDIAN) | tr '[A-Z]' '[a-z]'`-endian \
 			-i $${inputfile} -f $${charmap} \
 			$${locale} ; \
 	done
Signature · 1 line
1.7.0.4
#1 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

Quoted 13 lines · Thierry Bultel

From: tbultel <[email protected]>

When calling 'tr' without quoting braces, bash can make really weird things if there are existing 'single-letter-named' directories
eg:
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
aaa
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
AAA

The (quick) analysis is that the callee (tr) argvs then contain 'm' thus the translation does not work

Quoted 6 lines

Using quotes works around it
tbultel at laois:~/test$ echo AAA | tr '[A-Z]' '[a-z]' aaa

This should be real name, not a username.

Quoted 9 lines

---
Makefile | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +0 -0
index 1496bd7..7f835c0 100644

Makefile target-generatelocales: host-localedef · line 554

Quoted 8 lines
Makefile +1 -1
@@ -548,7 +548,7 @@ target-generatelocales: host-localedef
 		I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
 		$(HOST_DIR)/usr/bin/localedef \
 			--prefix=$(TARGET_DIR) \
-			--`echo $(BR2_ENDIAN) | tr [A-Z] [a-z]`-endian \
+			--`echo $(BR2_ENDIAN) | tr '[A-Z]' '[a-z]'`-endian \
 			-i $${inputfile} -f $${charmap} \
 			$${locale} ; \
 	done

While there's nothing wrong with your fix itself, I wonder why this isn't using the UPPERCASE and LOWERCASE macros we have defined in package/pkg-utils.mak...

Best regards,
Thomas

#2 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

Le 09/11/2013 14:47, Thomas De Schampheleire a ?crit :

Quoted 43 lines
Quoted 13 lines · Thierry Bultel

From: tbultel <[email protected]>

When calling 'tr' without quoting braces, bash can make really weird things if there are existing 'single-letter-named' directories
eg:
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
aaa
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
AAA

The (quick) analysis is that the callee (tr) argvs then contain 'm' thus the translation does not work

Quoted 6 lines

Using quotes works around it
tbultel at laois:~/test$ echo AAA | tr '[A-Z]' '[a-z]' aaa

This should be real name, not a username.

Quoted 9 lines

---
Makefile | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +0 -0
index 1496bd7..7f835c0 100644

Makefile target-generatelocales: host-localedef · line 554

Quoted 8 lines
Makefile +1 -1
@@ -548,7 +548,7 @@ target-generatelocales: host-localedef
 		I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
 		$(HOST_DIR)/usr/bin/localedef \
 			--prefix=$(TARGET_DIR) \
-			--`echo $(BR2_ENDIAN) | tr [A-Z] [a-z]`-endian \
+			--`echo $(BR2_ENDIAN) | tr '[A-Z]' '[a-z]'`-endian \
 			-i $${inputfile} -f $${charmap} \
 			$${locale} ; \
 	done

While there's nothing wrong with your fix itself, I wonder why this isn't using the UPPERCASE and LOWERCASE macros we have defined in package/pkg-utils.mak...

Quoted 5 lines

Best regards,
Thomas

The LOWERCASE macro does not seem to exist

#3 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

Quoted 44 lines · Thierry Bultel

Le 09/11/2013 14:47, Thomas De Schampheleire a ?crit :

Quoted 43 lines
Quoted 13 lines · Thierry Bultel

From: tbultel <[email protected]>

When calling 'tr' without quoting braces, bash can make really weird things if there are existing 'single-letter-named' directories
eg:
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
aaa
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
AAA

The (quick) analysis is that the callee (tr) argvs then contain 'm' thus the translation does not work

Quoted 6 lines

Using quotes works around it
tbultel at laois:~/test$ echo AAA | tr '[A-Z]' '[a-z]' aaa

This should be real name, not a username.

Quoted 9 lines

---
Makefile | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +0 -0
index 1496bd7..7f835c0 100644

Makefile target-generatelocales: host-localedef · line 554

Quoted 8 lines
Makefile +1 -1
@@ -548,7 +548,7 @@ target-generatelocales: host-localedef
 		I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
 		$(HOST_DIR)/usr/bin/localedef \
 			--prefix=$(TARGET_DIR) \
-			--`echo $(BR2_ENDIAN) | tr [A-Z] [a-z]`-endian \
+			--`echo $(BR2_ENDIAN) | tr '[A-Z]' '[a-z]'`-endian \
 			-i $${inputfile} -f $${charmap} \
 			$${locale} ; \
 	done

While there's nothing wrong with your fix itself, I wonder why this isn't using the UPPERCASE and LOWERCASE macros we have defined in package/pkg-utils.mak...

Quoted 8 lines
Quoted 5 lines

Best regards,
Thomas

The LOWERCASE macro does not seem to exist

It should be straightforward to implement it based on the existing UPPERCASE macro. I justfind odd there'd be two ways of converting case. One with direct tr and the other with a make macro...

#4 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

Quoted 51 lines · Thomas De Schampheleire
Quoted 50 lines · Thierry Bultel

Le 09/11/2013 14:47, Thomas De Schampheleire a ?crit :

Quoted 48 lines
Quoted 13 lines · Thierry Bultel

From: tbultel <[email protected]>

When calling 'tr' without quoting braces, bash can make really weird things if there are existing 'single-letter-named' directories
eg:
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
aaa
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z]
AAA

The (quick) analysis is that the callee (tr) argvs then contain 'm' thus the translation does not work

Quoted 6 lines

Using quotes works around it
tbultel at laois:~/test$ echo AAA | tr '[A-Z]' '[a-z]' aaa

This should be real name, not a username.

Quoted 9 lines

---
Makefile | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +0 -0
index 1496bd7..7f835c0 100644

Makefile target-generatelocales: host-localedef · line 554

Quoted 8 lines
Makefile +1 -1
@@ -548,7 +548,7 @@ target-generatelocales: host-localedef
 		I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
 		$(HOST_DIR)/usr/bin/localedef \
 			--prefix=$(TARGET_DIR) \
-			--`echo $(BR2_ENDIAN) | tr [A-Z] [a-z]`-endian \
+			--`echo $(BR2_ENDIAN) | tr '[A-Z]' '[a-z]'`-endian \
 			-i $${inputfile} -f $${charmap} \
 			$${locale} ; \
 	done

While there's nothing wrong with your fix itself, I wonder why this isn't using the UPPERCASE and LOWERCASE macros we have defined in package/pkg-utils.mak...

Best regards,
Thomas

Quoted 4 lines
Quoted 2 lines

The LOWERCASE macro does not seem to exist

It should be straightforward to implement it based on the existing UPPERCASE macro. I justfind odd there'd be two ways of converting case. One with direct tr and the other with a make macro...

The UPPERCASE macro is actually really really ugly - it's only reason
of existing is that it's much faster than calling tr 40K times, which would be required because UPPERCASE is called so often.

Bottom line: I think using tr for this one situation is a better idea
than adding another ugly lowercase macro.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#5 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

Le 10/11/2013 17:15, Arnout Vandecappelle a ?crit :

Quoted 66 lines · Thomas De Schampheleire
Quoted 54 lines
Quoted 53 lines · Thierry Bultel

Le 09/11/2013 14:47, Thomas De Schampheleire a ?crit :

Quoted 51 lines
Quoted 14 lines · Thierry Bultel

From: tbultel <[email protected]>

When calling 'tr' without quoting braces, bash can make really weird things
if there are existing 'single-letter-named' directories eg:
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z] aaa
tbultel at laois:~/test$ echo AAA | tr [A-Z] [a-z] AAA

The (quick) analysis is that the callee (tr) argvs then contain 'm' thus the translation does not work

Quoted 6 lines

Using quotes works around it
tbultel at laois:~/test$ echo AAA | tr '[A-Z]' '[a-z]' aaa

This should be real name, not a username.

Quoted 9 lines

---
Makefile | 2 +-

1 files changed, 1 insertions(+), 1 deletions(-)
Makefile +0 -0
index 1496bd7..7f835c0 100644

Makefile target-generatelocales: host-localedef · line 554

Quoted 8 lines
Makefile +1 -1
@@ -548,7 +548,7 @@ target-generatelocales: host-localedef
        I18NPATH=$(STAGING_DIR)/usr/share/i18n:/usr/share/i18n \
        $(HOST_DIR)/usr/bin/localedef \
            --prefix=$(TARGET_DIR) \
-            --`echo $(BR2_ENDIAN) | tr [A-Z] [a-z]`-endian \
+            --`echo $(BR2_ENDIAN) | tr '[A-Z]' '[a-z]'`-endian \
            -i $${inputfile} -f $${charmap} \
            $${locale} ; \
    done

While there's nothing wrong with your fix itself, I wonder why this isn't using the UPPERCASE and LOWERCASE macros we have defined in package/pkg-utils.mak...

Best regards,
Thomas

Quoted 6 lines
Quoted 2 lines

The LOWERCASE macro does not seem to exist

It should be straightforward to implement it based on the existing UPPERCASE macro. I justfind odd there'd be two ways of converting case. One with direct tr and the other with a make macro...

The UPPERCASE macro is actually really really ugly - it's only reason
of existing is that it's much faster than calling tr 40K times, which would be required because UPPERCASE is called so often.

Quoted 7 lines

Bottom line: I think using tr for this one situation is a better idea
than adding another ugly lowercase macro.

Regards,
Arnout

Yes indeed, one annoying point would have been the non symetric implementation, since UPPERCASE handles dots and hyphens specially.

I then re-submit my patch with just my name instead my user name. Sorry about that, this is because I have several work locations.

Regards,
Thierry

#6 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

In reply to Arnout Vandecappelle
Quoted 3 lines · Arnout Vandecappelle
Quoted 2 lines · Thomas De Schampheleire

Thierry Bultel <[email protected]> wrote:

[..]

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

The LOWERCASE macro does not seem to exist

It should be straightforward to implement it based on the existing UPPERCASE macro. I justfind odd there'd be two ways of converting case. One with direct tr and the other with a make macro...

Quoted 6 lines

The UPPERCASE macro is actually really really ugly - it's only reason of
existing is that it's much faster than calling tr 40K times, which would be required because UPPERCASE is called so often.

Bottom line: I think using tr for this one situation is a better idea than
adding another ugly lowercase macro.

It's also to create a lowercase macro that does $(shell tr '[A-Z]' '[a-z]'), explaining that the 'ugly' non-tr implementation is not needed because lowercase isn't called that often.
The reason that I like the centralization is that you can make one correct implementation, and other people won't make the forgotten-quote mistake a second time.

Also, I think [A-Z] should be replaced by [:upper:] and [a-z] by [:lower:] as this is more generally correct.

Best regards,
Thomas

#7 lore raw

PATCH 1/1 Generation of locales: made call to tr more robust

Le 10/11/2013 18:01, Thomas De Schampheleire a ?crit :

Quoted 27 lines · Arnout Vandecappelle
Quoted 3 lines · Thomas De Schampheleire
Quoted 2 lines

Thierry Bultel <[email protected]> wrote:

[..]

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

The LOWERCASE macro does not seem to exist

It should be straightforward to implement it based on the existing UPPERCASE macro. I justfind odd there'd be two ways of converting case. One with direct tr and the other with a make macro...

Quoted 6 lines

The UPPERCASE macro is actually really really ugly - it's only reason of
existing is that it's much faster than calling tr 40K times, which would be required because UPPERCASE is called so often.

Bottom line: I think using tr for this one situation is a better idea than
adding another ugly lowercase macro.

It's also to create a lowercase macro that does $(shell tr '[A-Z]' '[a-z]'), explaining that the 'ugly' non-tr implementation is not needed because lowercase isn't called that often.
The reason that I like the centralization is that you can make one correct implementation, and other people won't make the forgotten-quote mistake a second time.

Quoted 5 lines

Also, I think [A-Z] should be replaced by [:upper:] and [a-z] by [:lower:] as this is more generally correct.

Best regards,
Thomas

Ok, please ignore the patch I just resent.

reworking ...

Quoted 1 line