new PRService features in meta

NOTE: This set of patches added supporting for new features in meta layer. It should work together with another branch of patches in bitbake.
git://git.pokylinux.org/poky-contrib llu/PRService_bitbake

It supports the export/import AUTOPR values from/to PRservice to lockdown the PR values for release. The typical usage scenario is:

1. UserA enabled the PRService and run buildings. The PRService will record the AUTOPR values(which will be included in the finale PKGR) for different packages the userA builds.

2. UserA runs "bitbake-prserv-tool export <file>" to export the AUTOPR values from the current PRService into an exported inc file.

3. UserB gets the exported inc file and runs the command
"bitbake -R <exported file> target" to lockdown and reproduce the same AUTOPR when generating packages.

4. UserB may run "bitbake-prserv-tool import <file>" to import the exported AUTOPR values back into his own PRService. The AUTOPR values will be incremented from there.

This set of changes also supports the per-recipe based PRSERV_HOST and PRSERV_PORT.

This set of changes makes the basichash instead of the basic as the default signature generator.

The following changes since commit 8f348ccad083d6c02c200652ff6295e701e88f0d:
Saul Wold (1):
coreutils: ensure --color works so DEPEND on libcap

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib llu/PRService_oecore
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=llu/PRService_oecore

Lianhao Lu (3):
meta/PRService: Added export/import fuctions. package.bbclass: per recipe PRSERV_HOST support conf/bitbake.conf: basichash as default signature.

 meta/classes/package.bbclass          |   23 ++++-
 meta/classes/prserv.bbclass           |  167 ++++++++++++++++++++++++++++++---
 meta/conf/bitbake.conf                |    6 +-
 meta/recipes-core/meta/prserv-misc.bb |   69 ++++++++++++++
 scripts/bitbake-prserv-tool           |   53 +++++++++++
 5 files changed, 299 insertions(+), 19 deletions(-)
 create mode 100644 meta/recipes-core/meta/prserv-misc.bb
 create mode 100755 scripts/bitbake-prserv-tool

[YOCTO #1556]
- Modified meta/class/package.bbclass and prserv.bbclass according to the change in PR service by adding PACKAGE_ARCH into the query tuple.

- Added new bitbake tasks export/import AUTOPR values from/to the PR service.

- Supported reading the AUTOPR values from the exported .inc file instead of reading it from remote PR service.

- Created a new script bitbake-prserv-tool to export/import the AUTOPR values from/to the PR service.

Typical usage scenario of the export/import is:
1. bitbake-prserv-tool export <file> to export the AUTOPR values from the current PR service into an exported .inc file.

2. Others may use that exported .inc file(to be included in the local.conf) to lockdown and reproduce the same AUTOPR when generating package feeds.

3. Others may "bitbake-prserv-tool import <file>" to import the AUTOPR values into their own PR service and the AUTOPR values will be incremented from there.

---
 meta/classes/package.bbclass          |   15 +++-
 meta/classes/prserv.bbclass           |  167 ++++++++++++++++++++++++++++++---
 meta/conf/bitbake.conf                |    4 +-
 meta/recipes-core/meta/prserv-misc.bb |   69 ++++++++++++++
 scripts/bitbake-prserv-tool           |   53 +++++++++++
 5 files changed, 290 insertions(+), 18 deletions(-)
 create mode 100644 meta/recipes-core/meta/prserv-misc.bb
 create mode 100755 scripts/bitbake-prserv-tool

index 9040eb4..07d6f9a 100644
@@ -351,10 +351,19 @@ def runtime_mapping_rename (varname, d):
 
 python package_get_auto_pr() {
 	if d.getVar('USE_PR_SERV', True) != "0":
-		auto_pr=prserv_get_pr_auto(d)
-		if auto_pr is None:
-			bb.fatal("Can NOT get auto PR revision from remote PR service")
+		try:
+			auto_pr=prserv_get_pr_auto(d)
+		except Exception as e:
+			bb.error(str(e))
+			raise bb.build.FuncFailed(e)
 			return
+		if auto_pr is None:
+			if d.getVar('PRSERV_LOCKDOWN', True):
+				bb.error("Can NOT get auto PRAUTO from lockdown exported file")
+			else:
+				bb.error("Can NOT get auto PRAUTO from remote PR service")
+			raise bb.build.FuncFailed(e)
+			return 
 		d.setVar('PRAUTO',str(auto_pr))
 }
 

index 1c21616..31a722e 100644
@@ -190,7 +190,7 @@ BP = "${BPN}-${PV}"
 #
 # network based PR service
 #
-USE_PR_SERV = "${@[1,0][(d.getVar('PRSERV_HOST',1) is None) or (d.getVar('PRSERV_PORT',1) is None)]}"
+USE_PR_SERV = "${@[1,0][((d.getVar('PRSERV_HOST',1) is None) or (d.getVar('PRSERV_PORT',1) is None)) and (d.getVar('PRSERV_LOCKDOWN',1) is None)]}"
 
 # Package info.
 

@@ -740,7 +740,7 @@ BB_CONSOLELOG = "${TMPDIR}/cooker.log.${DATETIME}"
 # Setup our default hash policy
 BB_SIGNATURE_HANDLER ?= "basic"
 BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
-BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE"
+BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST PRSERV_PORT PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN"
 
 MLPREFIX ??= ""
 MULTILIB_VARIANTS ??= ""

1.7.0.4

[YOCTO #1126]
Added per recipe PRSERV_HOST PRSERV_PORT support.

---
 meta/classes/package.bbclass |   10 +++++++++-
 meta/conf/bitbake.conf       |    2 +-
 2 files changed, 10 insertions(+), 2 deletions(-)

index 07d6f9a..4fb7cc1 100644
@@ -350,6 +350,14 @@ def runtime_mapping_rename (varname, d):
 #
 
 python package_get_auto_pr() {
+	# per recipe PRSERV_HOST PRSERV_PORT
+	pn = d.getVar('PN', True)
+	host = d.getVar("PRSERV_HOST_" + pn, True)
+	port = d.getVar("PRSERV_PORT_" + pn, True)
+	if not (host is None):
+		d.setVar("PRSERV_HOST", host)
+	if not (port is None):
+		d.setVar("PRSERV_PORT", port)
 	if d.getVar('USE_PR_SERV', True) != "0":
 		try:
 			auto_pr=prserv_get_pr_auto(d)

@@ -362,7 +370,7 @@ python package_get_auto_pr() {
 				bb.error("Can NOT get auto PRAUTO from lockdown exported file")
 			else:
 				bb.error("Can NOT get auto PRAUTO from remote PR service")
-			raise bb.build.FuncFailed(e)
+			raise bb.build.FuncFailed()
 			return 
 		d.setVar('PRAUTO',str(auto_pr))
 }

index 31a722e..6b1744f 100644
@@ -190,7 +190,7 @@ BP = "${BPN}-${PV}"
 #
 # network based PR service
 #
-USE_PR_SERV = "${@[1,0][((d.getVar('PRSERV_HOST',1) is None) or (d.getVar('PRSERV_PORT',1) is None)) and (d.getVar('PRSERV_LOCKDOWN',1) is None)]}"
+USE_PR_SERV = "${@[1,0][((d.getVar('PRSERV_HOST',1) is None) or (d.getVar('PRSERV_HOST',1) == "") or (d.getVar('PRSERV_PORT',1) is None) or (d.getVar('PRSERV_PORT',1) == "")) and (d.getVar('PRSERV_LOCKDOWN',1) is None)]}"
 
 # Package info.
 

1.7.0.4

[YOCTO #1555]
Make basichash as default signature handler

---
 meta/conf/bitbake.conf |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

index 6b1744f..eca37e5 100644
@@ -738,7 +738,7 @@ TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', True).replace("_", "-")}"
 BB_CONSOLELOG = "${TMPDIR}/cooker.log.${DATETIME}"
 
 # Setup our default hash policy
-BB_SIGNATURE_HANDLER ?= "basic"
+BB_SIGNATURE_HANDLER ?= "basichash"
 BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)"
 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST PRSERV_PORT PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN"
 

1.7.0.4

I hope someone have tried to do incremental builds for some bigger image (not just core-*) for multiple machines over few weeks, because my observation
http://www.mail-archive.com/[email protected]/msg11438.html
still holds.

Regards,

Martin 'JaMa' Jansa     jabber: [email protected]

Don't do this. I realize there are flawed examples of this elsewhere in the metadata, but this isn't what FuncFailed is intended for. It's what the bitbake exec_func raises when a function fails, not what a function it executes should be raising.

Christopher Larson