Buildhistory sorting fixes

PATCH In review Paul Eggleton 3 participants 7 messages Feb 7, 2012

lore

Share & tools

lore

Paul Eggleton

author

2012-02-07 16:43 UTC #0 lore raw

cover Buildhistory sorting fixes

The following changes since commit 18d9fcfc4bc4b01f73e89f3b988c9d3d543c7705:

libzypp: add missing runtime dependences on gzip and gnupg (2012-02-03 17:23:28 +0000)

are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/buildhistory-fix3
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/buildhistory-fix3

Paul Eggleton (2):
classes/buildhistory: sort image file list classes/buildhistory: sort list fields in package info

 meta/classes/buildhistory.bbclass |   21 ++++++++++++++++-----
 1 files changed, 16 insertions(+), 5 deletions(-)

Signature · 1 line

1.7.5.4

Paul Eggleton

author

2012-02-07 16:43 UTC #1 lore raw

PATCH 1/2 classes/buildhistory: sort image file list

Sort the contents of files-in-image.txt to avoid unnecessary changes showing up in the history due to reordering.

Signed-off-by Paul Eggleton <[email protected]>

---
 meta/classes/buildhistory.bbclass |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

meta/classes/buildhistory.bbclass +1 -1

index 69a9d02..3964247 100644
@@ -310,7 +310,7 @@ buildhistory_get_image_installed() {
 buildhistory_get_imageinfo() {
 	# List the files in the image, but exclude date/time etc.
 	# This awk script is somewhat messy, but handles where the size is not printed for device files under pseudo
-	( cd ${IMAGE_ROOTFS} && find . -ls | awk '{ if ( $7 ~ /[0-9]/ ) printf "%s %10-s %10-s %10s %s %s %s\n", $3, $5, $6, $7, $11, $12, $13 ; else printf "%s %10-s %10-s %10s %s %s %s\n", $3, $5, $6, 0, $10, $11, $12 }' > ${BUILDHISTORY_DIR_IMAGE}/files-in-image.txt )
+	( cd ${IMAGE_ROOTFS} && find . -ls | awk '{ if ( $7 ~ /[0-9]/ ) printf "%s %10-s %10-s %10s %s %s %s\n", $3, $5, $6, $7, $11, $12, $13 ; else printf "%s %10-s %10-s %10s %s %s %s\n", $3, $5, $6, 0, $10, $11, $12 }' | sort -k5 > ${BUILDHISTORY_DIR_IMAGE}/files-in-image.txt )
 
 	# Record some machine-readable meta-information about the image
 	echo -n > ${BUILDHISTORY_DIR_IMAGE}/image-info.txt

Signature · 1 line

1.7.5.4

Paul Eggleton

author

2012-02-07 16:43 UTC #2 lore raw

PATCH 2/2 classes/buildhistory: sort list fields in package info

In reply to Paul Eggleton

Sort DEPENDS, PACKAGES, RDEPENDS, and RRECOMMENDS in package info files so that any changes in order (which are not important) are smoothed out in the change history.

Fixes [YOCTO #1961]

Signed-off-by Paul Eggleton <[email protected]>

---
 meta/classes/buildhistory.bbclass |   19 +++++++++++++++----
 1 files changed, 15 insertions(+), 4 deletions(-)

meta/classes/buildhistory.bbclass +15 -4

index 3964247..26d1a91 100644
@@ -126,17 +126,28 @@ python buildhistory_emit_pkghistory() {
 	def squashspaces(string):
 		return re.sub("\s+", " ", string)
 
+	def sortpkglist(string):
+		pkgiter = re.finditer(r'[a-zA-Z0-9.-]+( \([><=]+ [^ )]+\))?', pkgs, 0)
+		pkglist = [p.group(0) for p in pkgiter]
+		pkglist.sort()
+		return ' '.join(pkglist)
+
+	def sortlist(string):
+		items = string.split(' ')
+		items.sort()
+		return ' '.join(items)
+
 	pn = d.getVar('PN', True)
 	pe = d.getVar('PE', True) or "0"
 	pv = d.getVar('PV', True)
 	pr = d.getVar('PR', True)
-	packages = squashspaces(d.getVar('PACKAGES', True))
+	packages = sortlist(squashspaces(d.getVar('PACKAGES', True)))
 
 	rcpinfo = RecipeInfo(pn)
 	rcpinfo.pe = pe
 	rcpinfo.pv = pv
 	rcpinfo.pr = pr
-	rcpinfo.depends = squashspaces(d.getVar('DEPENDS', True) or "")
+	rcpinfo.depends = sortlist(squashspaces(d.getVar('DEPENDS', True) or ""))
 	rcpinfo.packages = packages
 	write_recipehistory(rcpinfo, d)
 	write_latestlink(None, pe, pv, pr, d)

@@ -164,8 +175,8 @@ python buildhistory_emit_pkghistory() {
 		pkginfo.pe = pe
 		pkginfo.pv = pv
 		pkginfo.pr = pr
-		pkginfo.rdepends = squashspaces(getpkgvar(pkg, 'RDEPENDS') or "")
-		pkginfo.rrecommends = squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or "")
+		pkginfo.rdepends = sortpkglist(squashspaces(getpkgvar(pkg, 'RDEPENDS') or ""))
+		pkginfo.rrecommends = sortpkglist(squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or ""))
 		pkginfo.files = squashspaces(getpkgvar(pkg, 'FILES') or "")
 
 		# Gather information about packaged files

Signature · 1 line

1.7.5.4

Koen Kooi

2012-02-07 17:58 UTC #3 lore raw

Re: [PATCH 2/2] classes/buildhistory: sort list fields in package info

Op 7 feb. 2012, om 17:25 heeft Paul Eggleton het volgende geschreven:

Quoted 12 lines

Sort DEPENDS, PACKAGES, RDEPENDS, and RRECOMMENDS in package info files so that any changes in order (which are not important) are smoothed out in the change history.
Fixes [YOCTO #1961]
Signed-off-by Paul Eggleton <[email protected]>
---
meta/classes/buildhistory.bbclass | 19 +++++++++++++++----
1 files changed, 15 insertions(+), 4 deletions(-)

meta/classes/buildhistory.bbclass python buildhistory_emit_pkghistory() { · line 144

Quoted 3 lines

meta/classes/buildhistory.bbclass +1 -1

 
-	packages = squashspaces(d.getVar('PACKAGES', True))
+	packages = sortlist(squashspaces(d.getVar('PACKAGES', True)))

Sorting PACKAGES is a bad idea, since order does matter in there.

Quoted 11 lines

rcpinfo = RecipeInfo(pn)
rcpinfo.pe = pe
rcpinfo.pv = pv
rcpinfo.pr = pr
meta/classes/buildhistory.bbclass +1 -1
-	rcpinfo.depends = squashspaces(d.getVar('DEPENDS', True) or "")
+	rcpinfo.depends = sortlist(squashspaces(d.getVar('DEPENDS', True) or ""))
rcpinfo.packages = packages
write_recipehistory(rcpinfo, d)
write_latestlink(None, pe, pv, pr, d)

meta/classes/buildhistory.bbclass python buildhistory_emit_pkghistory() { · line 179

Quoted 7 lines

meta/classes/buildhistory.bbclass +2 -2

@@ -164,8 +175,8 @@ python buildhistory_emit_pkghistory() {
 		pkginfo.pe = pe
 		pkginfo.pv = pv
 		pkginfo.pr = pr
-		pkginfo.rdepends = squashspaces(getpkgvar(pkg, 'RDEPENDS') or "")
-		pkginfo.rrecommends = squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or "")
+		pkginfo.rdepends = sortpkglist(squashspaces(getpkgvar(pkg, 'RDEPENDS') or ""))
+		pkginfo.rrecommends = sortpkglist(squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or ""))

Not sure what went wrong, but R* didn't seem to get sorted after these patches: https://github.com/Angstrom-distribution/buildhistory/commit/7a9f2f9308b468706799be18b3f400410ee1f664

regards,

Koen

Otavio Salvador

2012-02-07 18:00 UTC #4 lore raw

Re: [PATCH 2/2] classes/buildhistory: sort list fields in package info

Quoted 1 line · Koen Kooi

Sorting PACKAGES is a bad idea, since order does matter in there.

Agreed. Even installed depends might change depending on the installation order.

Signature · 3 lines

Otavio Salvador                             O.S. Systems
E-mail: [email protected]  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br

Koen Kooi

2012-02-07 18:00 UTC #5 lore raw

Re: [PATCH 2/2] classes/buildhistory: sort list fields in package info

In reply to Koen Kooi

Op 7 feb. 2012, om 18:50 heeft Koen Kooi het volgende geschreven:

Quoted 33 lines

Op 7 feb. 2012, om 17:25 heeft Paul Eggleton het volgende geschreven:
Quoted 12 lines
Sort DEPENDS, PACKAGES, RDEPENDS, and RRECOMMENDS in package info files so that any changes in order (which are not important) are smoothed out in the change history.
Fixes [YOCTO #1961]
Signed-off-by Paul Eggleton <[email protected]>
---
meta/classes/buildhistory.bbclass | 19 +++++++++++++++----
1 files changed, 15 insertions(+), 4 deletions(-)
meta/classes/buildhistory.bbclass python buildhistory_emit_pkghistory() { · line 144
Quoted 3 lines
meta/classes/buildhistory.bbclass +1 -1
 
-	packages = squashspaces(d.getVar('PACKAGES', True))
+	packages = sortlist(squashspaces(d.getVar('PACKAGES', True)))
Sorting PACKAGES is a bad idea, since order does matter in there.
Quoted 12 lines
rcpinfo = RecipeInfo(pn)
rcpinfo.pe = pe
rcpinfo.pv = pv
rcpinfo.pr = pr
meta/classes/buildhistory.bbclass +1 -1
-	rcpinfo.depends = squashspaces(d.getVar('DEPENDS', True) or "")
+	rcpinfo.depends = sortlist(squashspaces(d.getVar('DEPENDS', True) or ""))
rcpinfo.packages = packages
write_recipehistory(rcpinfo, d)
write_latestlink(None, pe, pv, pr, d)
meta/classes/buildhistory.bbclass +0 -0
@@ -164,8 +175,8 @@ python buildhistory_emit_pkghistory() {
 		pkginfo.pe = pe

Quoted 8 lines

meta/classes/buildhistory.bbclass python buildhistory_emit_pkghistory() { · line 179
Quoted 6 lines
meta/classes/buildhistory.bbclass +2 -2
 		pkginfo.pv = pv
 		pkginfo.pr = pr
-		pkginfo.rdepends = squashspaces(getpkgvar(pkg, 'RDEPENDS') or "")
-		pkginfo.rrecommends = squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or "")
+		pkginfo.rdepends = sortpkglist(squashspaces(getpkgvar(pkg, 'RDEPENDS') or ""))
+		pkginfo.rrecommends = sortpkglist(squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or ""))
Not sure what went wrong, but R* didn't seem to get sorted after these patches: https://github.com/Angstrom-distribution/buildhistory/commit/7a9f2f9308b468706799be18b3f400410ee1f664

*blush* checking the console showed the error;

ERROR: Error executing a python function in /OE/tentacle/sources/meta-openembedded/meta-oe/recipes-core/systemd/systemd_git.bb:
NameError: global name 'pkgs' is not defined

ERROR: The stack trace of python calls that resulted in this exception/failure was:
ERROR: File "buildhistory_emit_pkghistory", line 176, in <module>
ERROR:
ERROR: File "buildhistory_emit_pkghistory", line 155, in buildhistory_emit_pkghistory
ERROR:
ERROR: File "buildhistory_emit_pkghistory", line 107, in sortpkglist
ERROR:

ERROR: The code that was being executed was:
ERROR:      0172:
ERROR:      0173:               write_latestlink(pkg, pe, pv, pr, d)
ERROR:      0174:
ERROR:      0175:
ERROR:  *** 0176:buildhistory_emit_pkghistory(d)
ERROR:      0177:
ERROR: (file: 'buildhistory_emit_pkghistory', lineno: 176, function: <module>)
ERROR:      0151:               pkginfo = PackageInfo(pkg)
ERROR:      0152:               pkginfo.pe = pe
ERROR:      0153:               pkginfo.pv = pv
ERROR:      0154:               pkginfo.pr = pr
ERROR:  *** 0155:               pkginfo.rdepends = sortpkglist(squashspaces(getpkgvar(pkg, 'RDEPENDS') or ""))
ERROR:      0156:               pkginfo.rrecommends = sortpkglist(squashspaces(getpkgvar(pkg, 'RRECOMMENDS') or ""))
ERROR:      0157:               pkginfo.files = squashspaces(getpkgvar(pkg, 'FILES') or "")
ERROR:      0158:
ERROR:      0159:               # Gather information about packaged files
ERROR: (file: 'buildhistory_emit_pkghistory', lineno: 155, function: buildhistory_emit_pkghistory)
ERROR: Function failed: buildhistory_emit_pkghistory
ERROR: Logfile of failure stored in: /OE/tentacle/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/systemd-git-r15/temp/log.do_package.14521
NOTE: package systemd-git-r15: task do_package: Failed
ERROR: Task 9 (/OE/tentacle/sources/meta-openembedded/meta-oe/recipes-core/systemd/systemd_git.bb, do_package) failed with exit code '1'
NOTE: Tasks Summary: Attempted 2635 tasks of which 2626 didn't need to be rerun and 1 failed.

+ cd /OE/tentacle/build/tmp-angstrom_2010_x-eglibc/work/armv7a-angstrom-linux-gnueabi/bblayers-1.0-r0/bblayers-1.0
+ buildhistory_commit
+ '[' '!' -d /data/ssd/OE/buildhistory/ ']'
+ cd /data/ssd/OE/buildhistory//
+ '[' '!' -d .git ']'
++ git status --porcelain
+ repostatus=' M packages/armv7a-angstrom-linux-gnueabi/systemd/latest'
+ '[' ' M packages/armv7a-angstrom-linux-gnueabi/systemd/latest' '!=' '' ']'
+ git add /data/ssd/OE/buildhistory//README /data/ssd/OE/buildhistory//images /data/ssd/OE/buildhistory//packages
++ cat /etc/hostname
+ HOSTNAME=dominion
+ git commit /data/ssd/OE/buildhistory// -m 'Build Angstrom v2012.02-core of angstrom v2012.02-core for machine beagleboard on dominion' --author 'Koen Kooi <[email protected]>'
+ '[' [email protected]:Angstrom-distribution/buildhistory.git '!=' '' ']'
+ git push -q [email protected]:Angstrom-distribution/buildhistory.git

Paul Eggleton

author

2012-02-07 18:45 UTC #6 lore raw

Re: [PATCH 2/2] classes/buildhistory: sort list fields in package info

Quoted 1 line · On Tuesday 07 February 2012 18:52:44 Koen Kooi wrote:

*blush* checking the console showed the error;

Whoops. You know I really did test this patch; I'm unsure as to how I did not get this error because the patch is definitely wrong.

Not sure what I was thinking on PACKAGES sorting either. Will submit a v2.

Cheers,
Paul

Signature · 2 lines

Paul Eggleton
Intel Open Source Technology Centre