Skip to contentSkip to conversation

libc-package.bbclass: Use install utility to create directories with right permissions

PATCH Peter Kjellerstedt 2 participants 4 messages Feb 14, 2019 – Feb 17, 2019

lore
Share & tools
lore
#0 lore raw
Quoted 21 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Khem Raj
Sent: den 12 februari 2019 06:48
To: [email protected]
Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

Could fix issues where dirs with wrong permission is created with mkdir -p

---
 meta/classes/libc-package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/libc- renamed from meta/classes/libc-package.bbclass +0 -0

package.bbclass
index 0b4c666a74..34025635ce 100644

meta/classes/libc-package.bbclass LOCALETREESRC ?= "${PKGD}" · line 64

Quoted 5 lines
meta/classes/libc-package.bbclass +1 -1
@@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
 do_prep_locale_tree() {
 	treedir=${WORKDIR}/locale-tree
 	rm -rf $treedir
-	mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
+	install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}

This does not change anything (install -d and mkdir -p does the same thing).

Quoted 6 lines
meta/classes/libc-package.bbclass +0 -0
 	tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
 	# unzip to avoid parsing errors
 	for i in $treedir/${datadir}/i18n/charmaps/*gz; do
		gunzip $i
	done
	tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}

This is where the problem happens. When the files are copied from ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that of ${LOCALETREESRC}${localedir}. However, I now realized that the problem may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES, which causes the above line to copy things it was never meant to copy, in this case ${LOCALETREESRC}${localedir}. I tried to just remove the line altogether and then it built just fine. But I do not know what the consequences are of removing it... I guess it is related to setting up an environment for qemu to run in, but since that seems to work fine without the files copied by the above line, maybe it is not needed? Or maybe it is just not needed when having usrmerge in DISTRO_FEATURES? I tried to backtrack it through the commit logs, but it was introduced with the do_prep_locale_tree() function, so no help there.

Quoted 2 lines

--
2.20.1

//Peter

#1 lore raw
Quoted 37 lines · Peter Kjellerstedt
Quoted 22 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Khem Raj
Sent: den 12 februari 2019 06:48
To: [email protected]
Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

Could fix issues where dirs with wrong permission is created with mkdir -p

---
 meta/classes/libc-package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/libc- renamed from meta/classes/libc-package.bbclass +0 -0

package.bbclass
index 0b4c666a74..34025635ce 100644

meta/classes/libc-package.bbclass LOCALETREESRC ?= "${PKGD}" · line 64

Quoted 5 lines
meta/classes/libc-package.bbclass +1 -1
@@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
 do_prep_locale_tree() {
      treedir=${WORKDIR}/locale-tree
      rm -rf $treedir
-     mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
+     install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}

This does not change anything (install -d and mkdir -p does the same thing).

Quoted 6 lines
meta/classes/libc-package.bbclass +0 -0
      tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
      # unzip to avoid parsing errors
      for i in $treedir/${datadir}/i18n/charmaps/*gz; do
              gunzip $i
      done
      tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}
Quoted 14 lines

This is where the problem happens. When the files are copied from ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that of ${LOCALETREESRC}${localedir}. However, I now realized that the problem may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES, which causes the above line to copy things it was never meant to copy, in this case ${LOCALETREESRC}${localedir}. I tried to just remove the line altogether and then it built just fine. But I do not know what the consequences are of removing it... I guess it is related to setting up an environment for qemu to run in, but since that seems to work fine without the files copied by the above line, maybe it is not needed? Or maybe it is just not needed when having usrmerge in DISTRO_FEATURES? I tried to backtrack it through the commit logs, but it was introduced with the do_prep_locale_tree() function, so no help there.

I wonder if we should use --preserve option to tar -xf call here for making it explicit.

Quoted 5 lines
Quoted 2 lines

--
2.20.1

//Peter

#2 lore raw
Quoted 8 lines

-----Original Message-----
From: Khem Raj <[email protected]>
Sent: den 14 februari 2019 16:56
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

Quoted 48 lines
Quoted 3 lines · Peter Kjellerstedt
Quoted 3 lines

-----Original Message-----
From: [email protected]

<openembedded-

Quoted 42 lines
Quoted 18 lines

[email protected]> On Behalf Of Khem Raj
Sent: den 12 februari 2019 06:48
To: [email protected]
Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

Could fix issues where dirs with wrong permission is created with mkdir -p

---
 meta/classes/libc-package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/libc-package.bbclass +0 -0
index 0b4c666a74..34025635ce 100644

meta/classes/libc-package.bbclass LOCALETREESRC ?= "${PKGD}" · line 64

Quoted 5 lines
meta/classes/libc-package.bbclass +1 -1
@@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
 do_prep_locale_tree() {
      treedir=${WORKDIR}/locale-tree
      rm -rf $treedir
-     mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
+     install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}

This does not change anything (install -d and mkdir -p does the same thing).

Quoted 6 lines
meta/classes/libc-package.bbclass +0 -0
      tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
      # unzip to avoid parsing errors
      for i in $treedir/${datadir}/i18n/charmaps/*gz; do
              gunzip $i
      done
      tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}

This is where the problem happens. When the files are copied from ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that of ${LOCALETREESRC}${localedir}. However, I now realized that the problem may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES, which causes the above line to copy things it was never meant to copy, in this case ${LOCALETREESRC}${localedir}. I tried to just remove the line altogether and then it built just fine. But I do not know what the consequences are of removing it...

It turned out that the consequence is that if one actually does use qemu to generate the locale files, then it fails without that line. However, since qemu is not used by default for this since quite a long time ago, it is not needed in the normal case.

Quoted 9 lines
Quoted 6 lines

I guess it is related to setting up an environment for qemu to run in, but since that seems to work fine without the files copied by the above line, maybe it is not needed? Or maybe it is just not needed when having usrmerge in DISTRO_FEATURES? I tried to backtrack it through the commit logs, but it was introduced with the do_prep_locale_tree() function, so no help there.

I wonder if we should use --preserve option to tar -xf call here for making it explicit.

I have added a pattern to the tar command that unpacks the files so only the files that are needed by qemu are unpacked (see the patch series I just sent). That should avoid the problem.

However, I wonder if anyone actually sets LOCALE_GENERATION_WITH_CROSS-LOCALEDEF to 0 (it is 1 by default), and uses qemu to generate the locale files? It takes quite a lot longer (four times on my machine, totaling nearly half an hour). Otherwise, if we removed that, the code could be simplified and avoid the copying of these files in the first place.

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

--
2.20.1

//Peter

//Peter

#3 lore raw
Quoted 77 lines · Peter Kjellerstedt
Quoted 9 lines

-----Original Message-----
From: Khem Raj <[email protected]>
Sent: den 14 februari 2019 16:56
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

Quoted 48 lines
Quoted 3 lines · Peter Kjellerstedt
Quoted 3 lines

-----Original Message-----
From: [email protected]

<openembedded-

Quoted 42 lines
Quoted 18 lines

[email protected]> On Behalf Of Khem Raj
Sent: den 12 februari 2019 06:48
To: [email protected]
Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

Could fix issues where dirs with wrong permission is created with mkdir -p

---
 meta/classes/libc-package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/libc-package.bbclass +0 -0
index 0b4c666a74..34025635ce 100644

meta/classes/libc-package.bbclass LOCALETREESRC ?= "${PKGD}" · line 64

Quoted 5 lines
meta/classes/libc-package.bbclass +1 -1
@@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
 do_prep_locale_tree() {
      treedir=${WORKDIR}/locale-tree
      rm -rf $treedir
-     mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
+     install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}

This does not change anything (install -d and mkdir -p does the same thing).

Quoted 6 lines
meta/classes/libc-package.bbclass +0 -0
      tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
      # unzip to avoid parsing errors
      for i in $treedir/${datadir}/i18n/charmaps/*gz; do
              gunzip $i
      done
      tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}

This is where the problem happens. When the files are copied from ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that of ${LOCALETREESRC}${localedir}. However, I now realized that the problem may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES, which causes the above line to copy things it was never meant to copy, in this case ${LOCALETREESRC}${localedir}. I tried to just remove the line altogether and then it built just fine. But I do not know what the consequences are of removing it...

It turned out that the consequence is that if one actually does use qemu to generate the locale files, then it fails without that line. However, since qemu is not used by default for this since quite a long time ago, it is not needed in the normal case.

Quoted 9 lines
Quoted 6 lines

I guess it is related to setting up an environment for qemu to run in, but since that seems to work fine without the files copied by the above line, maybe it is not needed? Or maybe it is just not needed when having usrmerge in DISTRO_FEATURES? I tried to backtrack it through the commit logs, but it was introduced with the do_prep_locale_tree() function, so no help there.

I wonder if we should use --preserve option to tar -xf call here for making it explicit.

I have added a pattern to the tar command that unpacks the files so only the files that are needed by qemu are unpacked (see the patch series I just sent). That should avoid the problem.

Quoted 6 lines

However, I wonder if anyone actually sets LOCALE_GENERATION_WITH_CROSS-LOCALEDEF to 0 (it is 1 by default), and uses qemu to generate the locale files? It takes quite a lot longer (four times on my machine, totaling nearly half an hour). Otherwise, if we removed that, the code could be simplified and avoid the copying of these files in the first place.

I think the option to not use cross-localedef should be kept alive, even if its slower, its
helpful in cross validating issues when we might have them with cross localedef

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

--
2.20.1

//Peter

//Peter