Skip to contentSkip to conversation

glibc-locale: Rewrite do_install using install utility instead of cp

PATCH In review Khem Raj 2 participants 4 messages Feb 8, 2019 – Feb 10, 2019

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 glibc-locale: Rewrite do_install using install utility instead of cp

This has been a constant source of trouble for build failures due to host-user-contaminated QA errors of sort

ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by uid 3004, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]

So far we have tried to mould cp command into not carrying the build user permissions into install area but it is never entirely fixed since the issue keeps popping up in various scenes

This patch replaces use of cp with install utility and specifies install mode for files explcitly

---
 meta/recipes-core/glibc/glibc-locale.inc | 41 ++++++++++++------------
 1 file changed, 20 insertions(+), 21 deletions(-)
meta/recipes-core/glibc/glibc-locale.inc +20 -21
index 6384f9cbf1..9cce61bf0b 100644
@@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
 
 LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
 
-do_install () {
-	mkdir -p ${D}${bindir} ${D}${datadir}
-	if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
-		cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
-	fi
-	if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
-		mkdir -p ${D}${localedir}
-		cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
-	fi
-	if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
-		mkdir -p ${D}${libdir}
-		if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
-			cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
-		fi
-		if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
-			cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
-		fi
-	fi
-	if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
-		cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
+copy_locale_files() {
+	local dir=$1 mode=$2
+
+	[ -e "${LOCALETREESRC}$dir" ] || return 0
+
+	for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
+		install -d ${D}$dir/$d
+		find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
+		-exec install -m $mode -t "${D}$dir/$d" {} \;
+	done
+}
+
+do_install() {
+	copy_locale_files ${bindir} 0755
+	copy_locale_files ${localedir} 0644
+	if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
+		copy_locale_files ${libdir}/gconv 0755
+		copy_locale_files ${datadir}/i18n 0644
 	fi
-	cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
+	copy_locale_files ${datadir}/locale 0644
+	install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
 }
 
 inherit libc-package
Signature · 1 line
2.20.1
#1 lore raw
Quoted 55 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Khem Raj
Sent: den 8 februari 2019 01:56
To: [email protected]
Subject: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

This has been a constant source of trouble for build failures due to host-user-contaminated QA errors of sort

ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca- es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by uid 3004, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]

So far we have tried to mould cp command into not carrying the build user permissions into install area but it is never entirely fixed since the issue keeps popping up in various scenes

This patch replaces use of cp with install utility and specifies install mode for files explcitly

---
 meta/recipes-core/glibc/glibc-locale.inc | 41 ++++++++++++------------
 1 file changed, 20 insertions(+), 21 deletions(-)
meta/recipes-core/glibc/glibc-locale.inc +0 -19
index 6384f9cbf1..9cce61bf0b 100644
@@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
 
 LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
 
-do_install () {
-	mkdir -p ${D}${bindir} ${D}${datadir}
-	if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
-		cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
-	fi
-	if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
-		mkdir -p ${D}${localedir}
-		cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
-	fi
-	if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
-		mkdir -p ${D}${libdir}
-		if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
-			cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
-		fi
-		if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
-			cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
-		fi
-	fi
-	if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef" · line 82

Quoted 10 lines
meta/recipes-core/glibc/glibc-locale.inc +9 -1
-		cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
+copy_locale_files() {
+	local dir=$1 mode=$2
+
+	[ -e "${LOCALETREESRC}$dir" ] || return 0
+
+	for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
+		install -d ${D}$dir/$d
+		find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
+		-exec install -m $mode -t "${D}$dir/$d" {} \;

I know I am a sucker for details, but please align the broken find line so that it is more obvious that the second line is a continuation, i.e.:

		find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
		     -exec install -m $mode -t "${D}$dir/$d" {} \;
Quoted 17 lines
meta/recipes-core/glibc/glibc-locale.inc +11 -2
+	done
+}
+
+do_install() {
+	copy_locale_files ${bindir} 0755
+	copy_locale_files ${localedir} 0644
+	if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
+		copy_locale_files ${libdir}/gconv 0755
+		copy_locale_files ${datadir}/i18n 0644
 	fi
-	cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
+	copy_locale_files ${datadir}/locale 0644
+	install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
 }
 
 inherit libc-package
--
Quoted 1 line

2.20.1

//Peter

author
#2 lore raw
Quoted 70 lines · Peter Kjellerstedt
Quoted 56 lines

-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Khem Raj
Sent: den 8 februari 2019 01:56
To: [email protected]
Subject: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

This has been a constant source of trouble for build failures due to host-user-contaminated QA errors of sort

ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca- es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by uid 3004, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]

So far we have tried to mould cp command into not carrying the build user permissions into install area but it is never entirely fixed since the issue keeps popping up in various scenes

This patch replaces use of cp with install utility and specifies install mode for files explcitly

---
 meta/recipes-core/glibc/glibc-locale.inc | 41 ++++++++++++------------
 1 file changed, 20 insertions(+), 21 deletions(-)
meta/recipes-core/glibc/glibc-locale.inc +0 -19
index 6384f9cbf1..9cce61bf0b 100644
@@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
 
 LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-locale"
 
-do_install () {
-     mkdir -p ${D}${bindir} ${D}${datadir}
-     if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
-             cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
-     fi
-     if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
-             mkdir -p ${D}${localedir}
-             cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
-     fi
-     if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
-             mkdir -p ${D}${libdir}
-             if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
-                     cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
-             fi
-             if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
-                     cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
-             fi
-     fi
-     if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef" · line 82

Quoted 10 lines
meta/recipes-core/glibc/glibc-locale.inc +9 -1
-             cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
+copy_locale_files() {
+     local dir=$1 mode=$2
+
+     [ -e "${LOCALETREESRC}$dir" ] || return 0
+
+     for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P '); do
+             install -d ${D}$dir/$d
+             find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
+             -exec install -m $mode -t "${D}$dir/$d" {} \;

I know I am a sucker for details, but please align the broken find line so that it is more obvious that the second line is a continuation, i.e.:

Quoted 3 lines
                find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
                     -exec install -m $mode -t "${D}$dir/$d" {} \;

You are doing good here, I really appreciate the feedback. Do you think aligning it
under beginning of quote is more readable or aligning where find cmd begins is better ?
I will make this as a followup patch and send it with next lot of minor changes that I have in
queue.

Quoted 21 lines
Quoted 17 lines
meta/recipes-core/glibc/glibc-locale.inc +11 -2
+     done
+}
+
+do_install() {
+     copy_locale_files ${bindir} 0755
+     copy_locale_files ${localedir} 0644
+     if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
+             copy_locale_files ${libdir}/gconv 0755
+             copy_locale_files ${datadir}/i18n 0644
      fi
-     cp -R --no-dereference --preserve=mode,links ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
+     copy_locale_files ${datadir}/locale 0644
+     install -m 0644 ${LOCALETREESRC}/SUPPORTED ${WORKDIR}/SUPPORTED
 }
 
 inherit libc-package
--
Quoted 1 line

2.20.1

//Peter

#3 lore raw
Quoted 101 lines

-----Original Message-----
From: Khem Raj <[email protected]>
Sent: den 8 februari 2019 16:21
To: Peter Kjellerstedt <[email protected]>
Cc: [email protected]
Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

Quoted 3 lines · Peter Kjellerstedt
Quoted 3 lines

-----Original Message-----
From: [email protected]

<openembedded-

Quoted 7 lines
Quoted 7 lines

[email protected]> On Behalf Of Khem Raj
Sent: den 8 februari 2019 01:56
To: [email protected]
Subject: [OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

This has been a constant source of trouble for build failures due

to

Quoted 4 lines
Quoted 4 lines

host-user-contaminated QA errors of sort

ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca- es+valencia/usr/lib/locale/ca_ES@valencia/LC_MONETARY is owned by

uid

Quoted 1 line
Quoted 1 line

3004, which is the same as the user running bitbake. This may be

due to

Quoted 3 lines
Quoted 3 lines

host contamination [host-user-contaminated]

So far we have tried to mould cp command into not carrying the

build

Quoted 1 line
Quoted 1 line

user permissions into install area but it is never entirely fixed

since

Quoted 3 lines
Quoted 3 lines

the issue keeps popping up in various scenes

This patch replaces use of cp with install utility and specifies

install

Quoted 5 lines
Quoted 5 lines

mode for files explcitly

---
 meta/recipes-core/glibc/glibc-locale.inc | 41 ++++++++++++--------

----

Quoted 3 lines

Review

Quoted 3 lines
 1 file changed, 20 insertions(+), 21 deletions(-)
(quoted diff) +0 -0

b/meta/recipes-core/glibc/glibc-locale.inc

Quoted 6 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef" · line 72

Quoted 6 lines

index 6384f9cbf1..9cce61bf0b 100644

meta/recipes-core/glibc/glibc-locale.inc +0 -0
@@ -71,28 +71,27 @@ FILES_localedef = "${bindir}/localedef"
 
 LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-

locale"

Quoted 5 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 5 lines
meta/recipes-core/glibc/glibc-locale.inc +0 -4
-do_install () {
-     mkdir -p ${D}${bindir} ${D}${datadir}
-     if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
-             cp -R --no-dereference --preserve=mode,links

${LOCALETREESRC}/${bindir}/* ${D}${bindir}

Quoted 4 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 4 lines
meta/recipes-core/glibc/glibc-locale.inc +0 -4
-     fi
-     if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
-             mkdir -p ${D}${localedir}
-             cp -R --no-dereference --preserve=mode,links

${LOCALETREESRC}/${localedir}/* ${D}${localedir}

Quoted 5 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 5 lines
meta/recipes-core/glibc/glibc-locale.inc +0 -5
-     fi
-     if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
-             mkdir -p ${D}${libdir}
-             if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
-                     cp -R --no-dereference --preserve=mode,links

${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}

Quoted 3 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 3 lines
meta/recipes-core/glibc/glibc-locale.inc +0 -3
-             fi
-             if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
-                     cp -R --no-dereference --preserve=mode,links

${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}

Quoted 4 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 4 lines
meta/recipes-core/glibc/glibc-locale.inc +0 -4
-             fi
-     fi
-     if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
-             cp -R --no-dereference --preserve=mode,links

${LOCALETREESRC}/${datadir}/locale ${D}${datadir}

Quoted 6 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 6 lines
meta/recipes-core/glibc/glibc-locale.inc +6 -0
+copy_locale_files() {
+     local dir=$1 mode=$2
+
+     [ -e "${LOCALETREESRC}$dir" ] || return 0
+
+     for d in . $(find "${LOCALETREESRC}$dir" -type d -printf '%P

'); do

Quoted 5 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef" · line 82

Quoted 3 lines
meta/recipes-core/glibc/glibc-locale.inc +3 -0
+             install -d ${D}$dir/$d
+             find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
+             -exec install -m $mode -t "${D}$dir/$d" {} \;

I know I am a sucker for details, but please align the broken find

line so

Quoted 5 lines

that it is more obvious that the second line is a continuation, i.e.:

                find "${LOCALETREESRC}$dir/$d" -maxdepth 1 -type f \
                     -exec install -m $mode -t "${D}$dir/$d" {} \;
Quoted 7 lines

You are doing good here, I really appreciate the feedback. Do you think aligning it
under beginning of quote is more readable or aligning where find cmd begins is better ?
I will make this as a followup patch and send it with next lot of minor changes that I have in
queue.

The patch, as it was integrated to master, is fine.

Quoted 12 lines
Quoted 11 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef"

Quoted 11 lines
meta/recipes-core/glibc/glibc-locale.inc +9 -1
+     done
+}
+
+do_install() {
+     copy_locale_files ${bindir} 0755
+     copy_locale_files ${localedir} 0644
+     if [ ${PACKAGE_NO_GCONV} -eq 0 ]; then
+             copy_locale_files ${libdir}/gconv 0755
+             copy_locale_files ${datadir}/i18n 0644
      fi
-     cp -R --no-dereference --preserve=mode,links

${LOCALETREESRC}/SUPPORTED ${WORKDIR}

Quoted 11 lines
Quoted 2 lines

meta/recipes-core/glibc/glibc-locale.inc FILES_localedef = "${bindir}/localedef" · line 93

Quoted 2 lines
meta/recipes-core/glibc/glibc-locale.inc +2 -0
+     copy_locale_files ${datadir}/locale 0644
+     install -m 0644 ${LOCALETREESRC}/SUPPORTED

${WORKDIR}/SUPPORTED

Quoted 8 lines
Quoted 5 lines
meta/recipes-core/glibc/glibc-locale.inc +0 -1
 }
 
 inherit libc-package
--

2.20.1

//Peter

//Peter