Skip to contentSkip to conversation

package.bbclass: copy dotfiles in root D to PKGD

PATCH Superseded Michael Smith 6 participants 9 messages Oct 8, 2010 – Oct 18, 2010

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 package.bbclass: copy dotfiles in root D to PKGD

Use shutil.copytree() to copy D ("image") to PKGD ("package"). The previous system("cp %s/* ...") missed dotfiles/dirs at the top-level.

---

I sent an earlier version of this a couple of weeks back, but it relied on shutil.copytree() functionality only available in Python 2.5+. This version should work on 2.4 (i.e. RHEL 5).

 classes/package.bbclass |   19 ++++++++++++-------
 1 files changed, 12 insertions(+), 7 deletions(-)
classes/package.bbclass +12 -7
index 35d9864..e7659ae 100644
@@ -354,15 +354,20 @@ python package_do_split_locales() {
 }
 
 python perform_packagecopy () {
-	dest = bb.data.getVar('D', d, True)
-	dvar = bb.data.getVar('PKGD', d, True)
+	import shutil
+
+	installdest = bb.data.getVar('D', d, True)
+	pkgcopy = bb.data.getVar('PKGD', d, True)
 
-	bb.mkdirhier(dvar)
+	# Start package population by taking a copy of the installed 
+	# files to operate on. Create missing parent directories of
+	# pkgcopy first (shutil.copytree() does this automatically but only
+	# in Python 2.5+).
+	bb.mkdirhier(pkgcopy)
+	shutil.rmtree(pkgcopy, True)
 
-	# Start by package population by taking a copy of the installed 
-	# files to operate on
-	os.system('rm -rf %s/*' % (dvar))
-	os.system('cp -pPR %s/* %s/' % (dest, dvar))
+	# Preserve symlinks.
+	shutil.copytree(installdest, pkgcopy, symlinks=True)
 }
 
 python populate_packages () {
Signature · 1 line
1.7.0.4
Acked-by on v2 1/1
#1 lore raw
Quoted 5 lines · Michael Smith

Use shutil.copytree() to copy D ("image") to PKGD ("package"). The previous system("cp %s/* ...") missed dotfiles/dirs at the top-level.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#2 lore raw

hmmm I guess I am late in responding anyway I want to request to revert this patch as it has already been pushed.
When compiling lirc do_package hangs forever the reason is that lirc has some files in its install tree which are named pipes and shutils.copy* functions do not consider special functions as I see in the documentation

http://docs.python.org/dev/library/shutil.html

and there is this bug report

http://bugs.python.org/issue3002

which clarifies it more which means unfortunately we have to keep using 'cp' in this case.

is revert ok ?

Thanks

-Khem

Quoted 21 lines · Chris Larson
Quoted 5 lines · Michael Smith

Use shutil.copytree() to copy D ("image") to PKGD ("package"). The previous system("cp %s/* ...") missed dotfiles/dirs at the top-level.

--
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
_______________________________________________
Openembedded-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

#3 lore raw
Quoted 4 lines
Quoted 4 lines · Michael Smith
Quoted 3 lines

Use shutil.copytree() to copy D ("image") to PKGD ("package"). The previous system("cp %s/* ...") missed dotfiles/dirs at the top-level.

I expect this problem could be fixed by adding an extra line, e.g. os.system('cp -pPR %s/.??* %s/' % (dest, dvar))

-Graham

#4 lore raw

2010/10/18 Graham Gower <[email protected]>:

Quoted 9 lines
Quoted 4 lines
Quoted 4 lines · Michael Smith
Quoted 3 lines

Use shutil.copytree() to copy D ("image") to PKGD ("package"). The previous system("cp %s/* ...") missed dotfiles/dirs at the top-level.

I expect this problem could be fixed by adding an extra line, e.g. os.system('cp -pPR %s/.??* %s/' % (dest, dvar))

-Graham

Do you mean as an addition to the copytree, or as an addition to the old code?

The patch we had reads:

python perform_packagecopy () {
- dest = bb.data.getVar('D', d, True)
- dvar = bb.data.getVar('PKGD', d, True)
+ import shutil
+
+ installdest = bb.data.getVar('D', d, True)
+ pkgcopy = bb.data.getVar('PKGD', d, True)

- bb.mkdirhier(dvar)
+ # Start package population by taking a copy of the installed
+ # files to operate on. Create missing parent directories of
+ # pkgcopy first (shutil.copytree() does this automatically but only
+ # in Python 2.5+).
+ bb.mkdirhier(pkgcopy)
+ shutil.rmtree(pkgcopy, True)

- # Start by package population by taking a copy of the installed
- # files to operate on
- os.system('rm -rf %s/*' % (dvar))
- os.system('cp -pPR %s/* %s/' % (dest, dvar))
+ # Preserve symlinks.
+ shutil.copytree(installdest, pkgcopy, symlinks=True)
}

Seems adding an additional os.system kind-a beats using the shutil.copytree as we then still have the os.system to do the cp I guess the best way forward is to revert the patch and extend the os.system call with your patter.
E.g. something like:
- os.system('cp -pPR %s/* %s/' % (dest, dvar))
+ os.system('cp -pPR %s/* %s/.??* %s/' % (dest, dest, dvar))

cp may have mutliple sources if the output is a dir.

Frans

#5 lore raw
Quoted 6 lines · Frans Meulenbroeks

2010/10/18 Graham Gower <[email protected]>:

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines · Michael Smith
Quoted 3 lines

Use shutil.copytree() to copy D ("image") to PKGD ("package"). The previous system("cp %s/* ...") missed dotfiles/dirs at the top-level.

Quoted 6 lines
Quoted 4 lines

I expect this problem could be fixed by adding an extra line, e.g. os.system('cp -pPR %s/.??* %s/' % (dest, dvar))

-Graham

Do you mean as an addition to the copytree, or as an addition to the old code?

I meant as an addition if/when the commit is reverted.

Quoted 9 lines

Seems adding an additional os.system kind-a beats using the shutil.copytree as we then still have the os.system to do the cp I guess the best way forward is to revert the patch and extend the os.system call with your patter.
E.g. something like:
-       os.system('cp -pPR %s/* %s/' % (dest, dvar))
+       os.system('cp -pPR %s/* %s/.??* %s/' % (dest, dest, dvar))

cp may have mutliple sources if the output is a dir.

Beautiful. I would give such a thing my ack.

-Graham

#6 lore raw
In reply to Frans Meulenbroeks

Review

Quoted 2 lines · Frans Meulenbroeks
(quoted diff) +1 -1
-	os.system('cp -pPR %s/* %s/' % (dest, dvar))
+	os.system('cp -pPR %s/* %s/.??* %s/' % (dest, dest, dvar))

why not "os.system('cp -pPR %s/. %s/' % (dest, dvar))"?

Enrico

#7 lore raw

2010/10/18 Enrico Scholz <[email protected]>:

Quoted 8 lines
Quoted 2 lines · Frans Meulenbroeks

-     os.system('cp -pPR %s/* %s/' % (dest, dvar))
+     os.system('cp -pPR %s/* %s/.??* %s/' % (dest, dest, dvar))

why not "os.system('cp -pPR %s/. %s/' % (dest, dvar))"?

Even better. Didn't think of that one.

#8 lore raw
In reply to Khem Raj
Quoted 4 lines · Khem Raj

which clarifies it more which means unfortunately we have to keep using 'cp' in this case.

is revert ok ?

Nice catch. I've reverted it and I'll send a new patch later today based on the discussion.

Mike