Skip to contentSkip to conversation

distrodata.bbcalss: To get more precise upstream version for pkg report system

PATCH In review Mei Lei 2 participants 6 messages Dec 22, 2010 – Dec 23, 2010

lore
Share & tools
lore

Mei Lei

author
#0 lore raw

cover distrodata.bbcalss: To get more precise upstream version for pkg report system

There are some changes to get more recipes upstream version and to generate some necessary data for pkg report system.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: lmei3/pkg-report-system
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=lmei3/pkg-report-system

Thanks,
Mei Lei <[email protected]>
---

Mei Lei (3):
distrodata.bbclass: Change for package report system base.bbclass: Add inherit information for checkpkg task bitbake.conf: Change the GNU_MIRROR to get more precise result

 meta/classes/base.bbclass       |    4 +-
 meta/classes/distrodata.bbclass |   44 +++++++++++++++++++++++++++++++-------
 meta/conf/bitbake.conf          |    2 +-
 3 files changed, 39 insertions(+), 11 deletions(-)
author
#1 lore raw

PATCH 1/3 distrodata.bbclass: Change for package report system

Add some code to generate data for package report system. Fix some bugs when checking upstrem version.

---
 meta/classes/distrodata.bbclass |   44 +++++++++++++++++++++++++++++++-------
 1 files changed, 36 insertions(+), 8 deletions(-)
meta/classes/distrodata.bbclass +36 -8
index 0378790..f4eb6fd 100644
@@ -1,4 +1,3 @@
-
 require conf/distro/include/distro_tracking_fields.inc
 
 addhandler distro_eventhandler
@@ -215,7 +214,6 @@ addhandler checkpkg_eventhandler
 python checkpkg_eventhandler() {
     from bb.event import Handled, NotHandled
     # if bb.event.getName(e) == "TaskStarted":
-
     if bb.event.getName(e) == "BuildStarted":
 	"""initialize log files."""
 	logpath = bb.data.getVar('LOG_DIR', e.data, 1)
@@ -234,6 +232,9 @@ python checkpkg_eventhandler() {
 	f.write("Package\tOwner\tURI Type\tVersion\tTracking\tUpstream\tTMatch\tRMatch\n")
         f.close()
         bb.utils.unlockfile(lf)
+	"""initialize log files for Package Report System"""
+	logpath2 = bb.data.getVar('LOG_DIR', e.data, 1)
+	bb.utils.mkdirhier(logpath2)
 
     return NotHandled
 }
@@ -244,7 +245,6 @@ python do_checkpkg() {
 	import sys
 	import re
 	import tempfile
-
 	"""
 	sanity check to ensure same name and type. Match as many patterns as possible
 	such as:
@@ -262,8 +262,8 @@ python do_checkpkg() {
 	prefix1 = "[a-zA-Z][a-zA-Z0-9]*([\-_][a-zA-Z]\w+)*[\-_]"	# match most patterns which uses "-" as separator to version digits
 	prefix2 = "[a-zA-Z]+"			# a loose pattern such as for unzip552.tar.gz
 	prefix = "(%s|%s)" % (prefix1, prefix2)
-	suffix = "(tar\.gz|tgz|tar\.bz2|zip)"
-	suffixtuple = ("tar.gz", "tgz", "zip", "tar.bz2")
+	suffix = "(tar\.gz|tgz|tar\.bz2|zip|tar\.xz)"
+	suffixtuple = ("tar.gz", "tgz", "zip", "tar.bz2", "tar.xz")
 
 	sinterstr = "(?P<name>%s?)(?P<ver>.*)" % prefix
 	sdirstr = "(?P<name>%s)(?P<ver>.*)\.(?P<type>%s$)" % (prefix, suffix)
@@ -327,7 +327,7 @@ python do_checkpkg() {
 			which is designed for check purpose but we override check command for our own purpose
 			"""
 			ld = bb.data.createCopy(d)
-			bb.data.setVar('CHECKCOMMAND_wget', "/usr/bin/env wget -t 1 --passive-ftp -O %s '${URI}'" \
+			bb.data.setVar('CHECKCOMMAND_wget', "/usr/bin/env wget -t 1 --passive-ftp -O %s --user-agent=\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.12) Gecko/20101027 Ubuntu/9.10 (karmic) Firefox/3.6.12\" '${URI}'" \
 					% tmpf.name, d)
 			bb.data.update_data(ld)
 
@@ -371,7 +371,7 @@ python do_checkpkg() {
 			else:
 				s = "(\d+[\.\-_])+\d+/?"
 				
-			searchstr = "[hH][rR][eE][fF]=\"%s\">" % s
+			searchstr = "[hH][rR][eE][fF]=\"%s\".*>" % s
 			reg = re.compile(searchstr)
 
 			valid = 0
@@ -425,7 +425,7 @@ python do_checkpkg() {
 			"""match "{PN}-5.21.1.tar.gz">{PN}-5.21.1.tar.gz """
 			pn1 = re.search("^%s" % prefix, curname).group()
 			s = "[^\"]*%s[^\d\"]*?(\d+[\.\-_])+[^\"]*" % pn1
-			searchstr = "[hH][rR][eE][fF]=\"%s\">" % s
+			searchstr = "[hH][rR][eE][fF]=\"%s\".*>" % s
 			reg = re.compile(searchstr)
 	
 			valid = 0
@@ -465,10 +465,30 @@ python do_checkpkg() {
 	bb.utils.mkdirhier(logpath)
 	logfile = os.path.join(logpath, "checkpkg.csv")
 
+	"""initialize log files for package report system"""
+	logpath2 = bb.data.getVar('LOG_DIR', d, 1)
+	bb.utils.mkdirhier(logpath2)
+	logfile2 = os.path.join(logpath2, "get_pkg_info.log.%s" % bb.data.getVar('PN', d, 1))
+	if not os.path.exists(logfile2):
+		slogfile = os.path.join(logpath2, "get_pkg_info.log")
+		if os.path.exists(slogfile):
+			os.remove(slogfile)
+		os.system("touch %s" % logfile2)
+		os.symlink(logfile2, slogfile)
+
 	"""generate package information from .bb file"""
 	pname = bb.data.getVar('PN', d, 1)
 	pdesc = bb.data.getVar('DESCRIPTION', d, 1)
 	pgrp = bb.data.getVar('SECTION', d, 1)
+	pversion = bb.data.getVar('PV', d, 1)
+	plicense = bb.data.getVar('LICENSE',d,1)
+	psection = bb.data.getVar('SECTION',d,1)
+	phome = bb.data.getVar('HOMEPAGE', d, 1)
+	prelease = bb.data.getVar('PR',d,1)
+	ppriority = bb.data.getVar('PRIORITY',d,1)
+	pdepends = bb.data.getVar('DEPENDS',d,1)
+	pbugtracker = bb.data.getVar('BUGTRACKER',d,1)
+	ppe = bb.data.getVar('PE',d,1)
 
 	found = 0
 	for uri in src_uri.split():
@@ -614,6 +634,14 @@ python do_checkpkg() {
 		  (pname, maintainer, pproto, pcurver, pmver, pupver, pmstatus, pstatus))
 	f.close()
 	bb.utils.unlockfile(lf)
+
+  	"""write into get_pkg_info log file to supply data for package report system"""
+	lf2 = bb.utils.lockfile(logfile2 + ".lock")
+	f2 = open(logfile2, "a")
+	f2.write("%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n" % \
+		  (pname,pversion,pupver,plicense,psection, phome,prelease, ppriority,pdepends,pbugtracker,ppe,pdesc,pstatus))
+	f2.close()
+	bb.utils.unlockfile(lf2)
 }
 
 addtask checkpkgall after do_checkpkg
Signature · 1 line
1.6.3.3
author
#3 lore raw

PATCH 3/3 bitbake.conf: Change the GNU_MIRROR to get more precise result

In reply to Mei Lei

Use this mirror, more result can be checked

---
 meta/conf/bitbake.conf |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
meta/conf/bitbake.conf +1 -1
index c26beea..67fc3fb 100644
@@ -502,7 +502,7 @@ FREESMARTPHONE_GIT = "git://git.freesmartphone.org"
 GENTOO_MIRROR = "http://distfiles.gentoo.org/distfiles"
 GNOME_GIT = "git://git.gnome.org"
 GNOME_MIRROR = "http://ftp.gnome.org/pub/GNOME/sources"
-GNU_MIRROR = "ftp://ftp.gnu.org/gnu"
+GNU_MIRROR = "ftp://ftp.gnu.org/pub/gnu"
 GPE_MIRROR = "http://gpe.linuxtogo.org/download/source"
 GPE_EXTRA_SVN = "svn://projects.linuxtogo.org/svn/gpe/trunk/extra;module=${PN}"
 GPE_SVN = "svn://projects.linuxtogo.org/svn/gpe/trunk/base;module=${PN}"
Signature · 1 line
1.6.3.3
author
#2 lore raw

PATCH 2/3 base.bbclass: Add inherit information for checkpkg task

In reply to Mei Lei

Add inherit distrodata for running checkpkg task

---
 meta/classes/base.bbclass |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
meta/classes/base.bbclass +2 -2
index c60048b..a8cf4fd 100644
@@ -2,12 +2,12 @@ BB_DEFAULT_TASK ?= "build"
 
 inherit patch
 inherit staging
-
 inherit mirrors
 inherit utils
 inherit utility-tasks
 inherit metadata_scm
-
+inherit distrodata
+ 
 python sys_path_eh () {
     if isinstance(e, bb.event.ConfigParsed):
         import sys
Signature · 1 line
1.6.3.3
#4 lore raw
In reply to Mei Lei
Quoted 10 lines · Mei Lei

There are some changes to get more recipes upstream version and to generate some necessary data for pkg report system.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: lmei3/pkg-report-system
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=lmei3/pkg-report-system

Thanks,
Mei Lei<[email protected]>
---

Lei:

There are various issues with these patches please see my comments below

Quoted 3 lines

Mei Lei (3):
distrodata.bbclass: Change for package report system

Not sure why you needed to add a logpath2 as you are using the same LOG_DIR var as logpath. Are you intending to create a separate directory.

Is there a reason you need to have each package into in separate files? Can your tool not read 1 larger flat file then we can just use 1 output format for the spreadsheet

Quoted 1 line

base.bbclass: Add inherit information for checkpkg task

We don't want to inherit the distrodata by default, you can add this to your local.conf

Quoted 2 lines

bitbake.conf: Change the GNU_MIRROR to get more precise result

Not sure how this gives a more precise result, gnu and pub/gnu are the same location, one is linked to the other.

Sau!

Quoted 10 lines
  meta/classes/base.bbclass       |    4 +-
  meta/classes/distrodata.bbclass |   44 +++++++++++++++++++++++++++++++-------
  meta/conf/bitbake.conf          |    2 +-
  3 files changed, 39 insertions(+), 11 deletions(-)

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

author
#5 lore raw
Quoted 25 lines

-----Original Message-----
From: Wold, Saul
Sent: Thursday, December 23, 2010 12:46 AM
To: Mei, Lei
Cc: [email protected]
Subject: Re: [poky] [PATCH 0/3]distrodata.bbcalss: To get more precise upstream version for pkg report system

Quoted 1 line · Mei Lei

There are some changes to get more recipes upstream version and to

generate some necessary data for pkg report system.

Quoted 4 lines

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: lmei3/pkg-report-system
Browse:

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=lmei3/pkg-report-system

Quoted 5 lines

Thanks,
Mei Lei<[email protected]>
---

Lei:

Quoted 7 lines

There are various issues with these patches please see my comments below

Quoted 3 lines

Mei Lei (3):
distrodata.bbclass: Change for package report system

Not sure why you needed to add a logpath2 as you are using the same LOG_DIR var as logpath. Are you intending to create a separate directory.

Oh, I didn't realize this problem before, I will change this.

Quoted 4 lines

Is there a reason you need to have each package into in separate files? Can your tool not read 1 larger flat file then we can just use 1 output format for the spreadsheet

Ok, I will change this output as one file.
The reason why I generate every output file for every package is that the default output this is it when I first use checkpkg task long long time ago, so the package report system obey this all the time. I will change this.

Quoted 4 lines
Quoted 2 lines

base.bbclass: Add inherit information for checkpkg task

We don't want to inherit the distrodata by default, you can add this to your local.conf

Ok, I understand it.

Quoted 6 lines
Quoted 3 lines

bitbake.conf: Change the GNU_MIRROR to get more precise result

Not sure how this gives a more precise result, gnu and pub/gnu are the same location, one is linked to the other.

Ok , I get it. This patch is not necessary.

Quoted 13 lines

Sau!

Quoted 2 lines
  meta/classes/base.bbclass       |    4 +-
  meta/classes/distrodata.bbclass |   44

+++++++++++++++++++++++++++++++-------

Quoted 8 lines
  meta/conf/bitbake.conf          |    2 +-
  3 files changed, 39 insertions(+), 11 deletions(-)

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky