new PRService features in meta
v3 cover new PRService features in meta
V3: Using event handler instead of new tasks to interfacing to bitbake.
V2: Using bb.fatal instead of bb.FuncFailed.
NOTE: This set of patches added supporting for new features in meta layer. It should work together with another branch of patches in bitbake.
git://git.pokylinux.org/poky-contrib llu/PRService_bitbake
It supports the export/import AUTOPR values from/to PRservice to lockdown the PR values for release. The typical usage scenario is:
1. UserA enabled the PRService and run buildings. The PRService will record the AUTOPR values(which will be included in the finale PKGR) for different packages the userA builds.
2. UserA runs "bitbake-prserv-tool export <file>" to export the AUTOPR values from the current PRService into an exported inc file.
3. UserB gets the exported inc file and runs the command
"bitbake -R <exported file> target" to lockdown and reproduce the same AUTOPR when generating packages.
4. UserB may run "bitbake-prserv-tool import <file>" to import the exported AUTOPR values back into his own PRService. The AUTOPR values will be incremented from there.
This set of changes also supports the per-recipe based PRSERV_HOST and PRSERV_PORT.
This set of changes makes the basichash instead of the basic as the default signature generator.
The following changes since commit fff18970706913e7fd7f4a119d798dddb44b388a:
Gary Thomas (1):
base-files: filesystems: fix mount order
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib llu/PRService_oecore
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=llu/PRService_oecore
Lianhao Lu (3):
meta/PRService: Added export/import fuctions. package.bbclass: per recipe PRSERV_HOST support conf/bitbake.conf: basichash as default signature.
meta/classes/package.bbclass | 21 +++++++- meta/classes/prexport.bbclass | 45 ++++++++++++++++ meta/classes/primport.bbclass | 17 ++++++ meta/classes/prserv.bbclass | 36 +++++-------- meta/conf/bitbake.conf | 6 +- meta/conf/prexport.conf | 1 + meta/conf/primport.conf | 1 + meta/lib/oe/prservice.py | 113 +++++++++++++++++++++++++++++++++++++++++ scripts/bitbake-prserv-tool | 57 +++++++++++++++++++++ 9 files changed, 269 insertions(+), 28 deletions(-) create mode 100644 meta/classes/prexport.bbclass create mode 100644 meta/classes/primport.bbclass create mode 100644 meta/conf/prexport.conf create mode 100644 meta/conf/primport.conf create mode 100644 meta/lib/oe/prservice.py create mode 100755 scripts/bitbake-prserv-tool
PATCH v3 1/3 meta/PRService: Added export/import fuctions.
[YOCTO #1556]
- Modified meta/class/package.bbclass and prserv.bbclass according to the change in PR service by adding PACKAGE_ARCH into the query tuple.
- Added prexport.bbclass, primport.bbclass to export/import AUTOPR values from/to PRService.
- Move PR service related common code to lib/oe/prservice.py.
- Supported reading the AUTOPR values from the exported .inc file instead of reading it from remote PR service.
- Created a new script bitbake-prserv-tool to export/import the AUTOPR values from/to the PR service.
Typical usage scenario of the export/import is:
1. bitbake-prserv-tool export <file> to export the AUTOPR values from the current PR service into an exported .inc file.
2. Others may use that exported .inc file(to be included in the local.conf) to lockdown and reproduce the same AUTOPR when generating package feeds.
3. Others may "bitbake-prserv-tool import <file>" to import the AUTOPR values into their own PR service and the AUTOPR values will be incremented from there.
--- meta/classes/package.bbclass | 13 ++++- meta/classes/prexport.bbclass | 45 ++++++++++++++++ meta/classes/primport.bbclass | 17 ++++++ meta/classes/prserv.bbclass | 36 +++++-------- meta/conf/bitbake.conf | 4 +- meta/conf/prexport.conf | 1 + meta/conf/primport.conf | 1 + meta/lib/oe/prservice.py | 113 +++++++++++++++++++++++++++++++++++++++++ scripts/bitbake-prserv-tool | 57 +++++++++++++++++++++ 9 files changed, 260 insertions(+), 27 deletions(-) create mode 100644 meta/classes/prexport.bbclass create mode 100644 meta/classes/primport.bbclass create mode 100644 meta/conf/prexport.conf create mode 100644 meta/conf/primport.conf create mode 100644 meta/lib/oe/prservice.py create mode 100755 scripts/bitbake-prserv-tool
meta/classes/package.bbclass +10 -3
@@ -351,10 +351,17 @@ def runtime_mapping_rename (varname, d): python package_get_auto_pr() { if d.getVar('USE_PR_SERV', True) != "0": - auto_pr=prserv_get_pr_auto(d) - if auto_pr is None: - bb.fatal("Can NOT get auto PR revision from remote PR service") + try: + auto_pr=prserv_get_pr_auto(d) + except Exception as e: + bb.fatal("Can NOT get PRAUTO, exception %s" % str(e)) return + if auto_pr is None: + if d.getVar('PRSERV_LOCKDOWN', True): + bb.fatal("Can NOT get PRAUTO from lockdown exported file") + else: + bb.fatal("Can NOT get PRAUTO from remote PR service") + return d.setVar('PRAUTO',str(auto_pr)) }
meta/classes/prexport.bbclass new file +45 -0
45 lines not on this page · raw message
meta/classes/primport.bbclass new file +17 -0
@@ -0,0 +1,17 @@ +python primport_handler () { + import bb.event + if not e.data: + return + + if isinstance(e, bb.event.ParseCompleted): + import oe.prservice + #import all exported AUTOPR values + imported = oe.prservice.prserv_import_db(e.data) + if imported is None: + bb.fatal("import failed!") + + for (version, pkgarch, checksum, value) in imported: + bb.note("imported (%s,%s,%s,%d)" % (version, pkgarch, checksum, value)) +} + +addhandler primport_handler
meta/classes/prserv.bbclass +14 -22
@@ -1,29 +1,21 @@ -def prserv_make_conn(d): - import prserv.serv - host=d.getVar("PRSERV_HOST",True) - port=d.getVar("PRSERV_PORT",True) - try: - conn=None - conn=prserv.serv.PRServerConnection(host,int(port)) - d.setVar("__PRSERV_CONN",conn) - except Exception, exc: - bb.fatal("Connecting to PR service %s:%s failed: %s" % (host, port, str(exc))) - - return conn - def prserv_get_pr_auto(d): - if d.getVar('USE_PR_SERV', True) != "0": + import oe.prservice + if d.getVar('USE_PR_SERV', True) != "1": bb.warn("Not using network based PR service") return None - conn=d.getVar("__PRSERV_CONN", True) - if conn is None: - conn=prserv_make_conn(d) + version = d.getVar("PRAUTOINX", True) + pkgarch = d.getVar("PACKAGE_ARCH", True) + checksum = d.getVar("BB_TASKHASH", True) + + if d.getVar('PRSERV_LOCKDOWN', True): + auto_rev = d.getVar('PRAUTO_' + version + '_' + pkgarch, True) or d.getVar('PRAUTO_' + version, True) or None + else: + conn = d.getVar("__PRSERV_CONN", True) if conn is None: - return None + conn = oe.prservice.prserv_make_conn(d) + if conn is None: + return None + auto_rev = conn.getPR(version, pkgarch, checksum) - version=d.getVar("PF", True) - checksum=d.getVar("BB_TASKHASH", True) - auto_rev=conn.getPR(version,checksum) - bb.debug(1,"prserv_get_pr_auto: version: %s checksum: %s result %d" % (version, checksum, auto_rev)) return auto_rev
meta/conf/bitbake.conf +2 -2
@@ -190,7 +190,7 @@ BP = "${BPN}-${PV}" # # network based PR service # -USE_PR_SERV = "${@[1,0][(d.getVar('PRSERV_HOST',1) is None) or (d.getVar('PRSERV_PORT',1) is None)]}" +USE_PR_SERV = "${@[1,0][((not d.getVar('PRSERV_HOST', True)) or (not d.getVar('PRSERV_PORT', True))) and (not d.getVar('PRSERV_LOCKDOWN', True))]}" # Package info.
@@ -733,7 +733,7 @@ BB_CONSOLELOG = "${TMPDIR}/cooker.log.${DATETIME}" # Setup our default hash policy BB_SIGNATURE_HANDLER ?= "basic" BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)" -BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE" +BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST PRSERV_PORT PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN" MLPREFIX ??= "" MULTILIB_VARIANTS ??= ""
meta/conf/prexport.conf new file +1 -0
@@ -0,0 +1 @@ +INHERIT += "prexport"
meta/conf/primport.conf new file +1 -0
@@ -0,0 +1 @@ +INHERIT += "primport"
meta/lib/oe/prservice.py new file +113 -0
113 lines not on this page · raw message
scripts/bitbake-prserv-tool new file +57 -0
57 lines not on this page · raw message
Signature · 1 line
1.7.0.4
PATCH v3 2/3 package.bbclass: per recipe PRSERV_HOST support
[YOCTO #1126]
Added per recipe PRSERV_HOST PRSERV_PORT support.
--- meta/classes/package.bbclass | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)
meta/classes/package.bbclass +8 -0
@@ -350,6 +350,14 @@ def runtime_mapping_rename (varname, d): # python package_get_auto_pr() { + # per recipe PRSERV_HOST PRSERV_PORT + pn = d.getVar('PN', True) + host = d.getVar("PRSERV_HOST_" + pn, True) + port = d.getVar("PRSERV_PORT_" + pn, True) + if not (host is None): + d.setVar("PRSERV_HOST", host) + if not (port is None): + d.setVar("PRSERV_PORT", port) if d.getVar('USE_PR_SERV', True) != "0": try: auto_pr=prserv_get_pr_auto(d)
Signature · 1 line
1.7.0.4
PATCH v3 3/3 conf/bitbake.conf: basichash as default signature.
In reply to Lianhao Lu[YOCTO #1555]
Make basichash as default signature handler
--- meta/conf/bitbake.conf | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
meta/conf/bitbake.conf +1 -1
@@ -731,7 +731,7 @@ TRANSLATED_TARGET_ARCH ??= "${@d.getVar('TARGET_ARCH', True).replace("_", "-")}" BB_CONSOLELOG = "${TMPDIR}/cooker.log.${DATETIME}" # Setup our default hash policy -BB_SIGNATURE_HANDLER ?= "basic" +BB_SIGNATURE_HANDLER ?= "basichash" BB_HASHTASK_WHITELIST ?= "(.*-cross$|.*-native$|.*-cross-initial$|.*-cross-intermediate$|^virtual:native:.*|^virtual:nativesdk:.*)" BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST PRSERV_PORT PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN"
Signature · 1 line
1.7.0.4
Quoted 35 lines · Lianhao Lu
V3: Using event handler instead of new tasks to interfacing to bitbake.
V2: Using bb.fatal instead of bb.FuncFailed.NOTE: This set of patches added supporting for new features in meta layer. It should work together with another branch of patches in bitbake.
git://git.pokylinux.org/poky-contribllu/PRService_bitbakeIt supports the export/import AUTOPR values from/to PRservice to lockdown the PR values for release. The typical usage scenario is:
1. UserA enabled the PRService and run buildings. The PRService will record the AUTOPR values(which will be included in the finale PKGR) for different packages the userA builds.
2. UserA runs "bitbake-prserv-tool export<file>" to export the AUTOPR values from the current PRService into an exported inc file.
3. UserB gets the exported inc file and runs the command
"bitbake -R<exported file> target" to lockdown and reproduce the same AUTOPR when generating packages.4. UserB may run "bitbake-prserv-tool import<file>" to import the exported AUTOPR values back into his own PRService. The AUTOPR values will be incremented from there.
This set of changes also supports the per-recipe based
PRSERV_HOSTandPRSERV_PORT.This set of changes makes the basichash instead of the basic as the default signature generator.
The following changes since commit
fff18970706913e7fd7f4a119d798dddb44b388a:
Gary Thomas (1):
base-files: filesystems: fix mount order
Quoted 8 lines
are available in the git repository at:
git://git.yoctoproject.org/poky-contribllu/PRService_oecore
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=llu/PRService_oecoreLianhao Lu (3):
meta/PRService: Added export/import fuctions. package.bbclass: per recipePRSERV_HOSTsupport
Merged these first 2 as infrastructure
Quoted 2 lines
conf/bitbake.conf: basichash as default signature.
Held off on this, please enable it in poky.conf first for more testing in poky first.
Thanks
Sau!
Quoted 23 lines
meta/classes/package.bbclass | 21 +++++++- meta/classes/prexport.bbclass | 45 ++++++++++++++++ meta/classes/primport.bbclass | 17 ++++++ meta/classes/prserv.bbclass | 36 +++++-------- meta/conf/bitbake.conf | 6 +- meta/conf/prexport.conf | 1 + meta/conf/primport.conf | 1 + meta/lib/oe/prservice.py | 113 +++++++++++++++++++++++++++++++++++++++++ scripts/bitbake-prserv-tool | 57 +++++++++++++++++++++ 9 files changed, 269 insertions(+), 28 deletions(-)create mode 100644 meta/classes/prexport.bbclass create mode 100644 meta/classes/primport.bbclass create mode 100644
meta/conf/prexport.confcreate mode 100644meta/conf/primport.confcreate mode 100644meta/lib/oe/prservice.pycreate mode 100755scripts/bitbake-prserv-tool_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core