denzil pull request 3
cover denzil pull request 3
Hi Richard,
This pull request correspons to the one I submitted to oe-core recently and that you've already merged, except for the two bitbake patches which we also need for the poky tree.
Scott
The following changes since commit e7d4eba0a9bc02d61a77129da4be2f0c9239884b:
Save proxy settings (2012-10-01 18:13:15 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib sgarman/denzil-next
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=sgarman/denzil-next-pull3
Darren Hart (1):
gnutls: Update SRC_URI to use GNU_MIRROR
Dennis Lan (1):
openjade-native: fix undefined Getopts error, use std namespace
Jesse Zhang (1):
udev: don't mount with -o sync
Martin Jansa (1):
openssl: add deprecated and unmaintained find.pl from perl-5.14 to
fix perlpath.pl
Paul Eggleton (3):
classes/cml1: ensure -c menuconfig forces a rebuild next time
bitbake: bitbake: ensure -f causes dependent tasks to be re-run
bitbake: lib/bb/siggen.py: log when tainting the signature of a task
Richard Purdie (1):
siteconfig: Clear cache before rebuilding
bitbake/lib/bb/build.py | 18 +++++++ bitbake/lib/bb/cooker.py | 6 +-- bitbake/lib/bb/runqueue.py | 12 ++--- bitbake/lib/bb/siggen.py | 35 +++++++++++++ meta/classes/cml1.bbclass | 16 ++++++ meta/classes/siteconfig.bbclass | 1 + .../openssl/openssl-1.0.0i/find.pl | 54 ++++++++++++++++++++ .../recipes-connectivity/openssl/openssl_1.0.0i.bb | 7 ++- meta/recipes-core/udev/udev/mount.sh | 2 +- meta/recipes-core/udev/udev_164.bb | 2 +- .../openjade/openjade-1.3.2/msggen.pl.patch | 44 ++++++++++++++++ .../openjade/openjade-native_1.3.2.bb | 3 +- meta/recipes-support/gnutls/gnutls.inc | 4 +- 13 files changed, 189 insertions(+), 15 deletions(-) create mode 100644 meta/recipes-connectivity/openssl/openssl-1.0.0i/find.pl create mode 100644 meta/recipes-devtools/openjade/openjade-1.3.2/msggen.pl.patch
Signature · 1 line
1.7.9.5
PATCH 1/8 udev: don't mount with -o sync
From: Jesse Zhang <[email protected]>
mount.sh mounts all partitions with -o sync, which is bad for system performance.
(From OE-Core rev: d49cf73754150b50a911d326aaa666f5da78855c)
--- meta/recipes-core/udev/udev/mount.sh | 2 +- meta/recipes-core/udev/udev_164.bb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/udev/udev/mount.sh +1 -1
@@ -23,7 +23,7 @@ automount() { ! test -d "/media/$name" && mkdir -p "/media/$name" - if ! $MOUNT -t auto -o sync $DEVNAME "/media/$name" + if ! $MOUNT -t auto $DEVNAME "/media/$name" then #logger "mount.sh/automount" "$MOUNT -t auto $DEVNAME \"/media/$name\" failed!" rm_dir "/media/$name"
meta/recipes-core/udev/udev_164.bb +1 -1
@@ -1,6 +1,6 @@ include udev.inc -PR = "r13" +PR = "r14" SRC_URI += "file://udev-166-v4l1-1.patch"
Signature · 1 line
1.7.9.5
PATCH 2/8 classes/cml1: ensure -c menuconfig forces a rebuild next time
In reply to Scott GarmanFrom: Paul Eggleton <[email protected]>
Ensure the following results in the kernel being rebuilt, repackaged and re-deployed in the final step:
bitbake virtual/kernel
bitbake -c menuconfig virtual/kernel
[ make changes to the kernel configuration and save ] bitbake virtual/kernel
If there are no changes to the configuration saved, the rebuild will not be triggered.
Note that this relies on a function recently added to BitBake and requires full hashing (i.e. BB_SIGNATURE_HANDLER must be set to a signature handler that inherits from BasicHash) - if this is not the case or the function is not available in the version of BitBake being used this change will do nothing.
Fixes [YOCTO #2256].
(From OE-Core rev: 9bf6b60e1599cf5dd87089d42584583cdfd6807a)
--- meta/classes/cml1.bbclass | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
meta/classes/cml1.bbclass +16 -0
@@ -10,7 +10,23 @@ addtask configure after do_unpack do_patch before do_compile inherit terminal python do_menuconfig() { + try: + mtime = os.path.getmtime(".config") + except OSError: + mtime = 0 + oe_terminal("make menuconfig", '${PN} Configuration', d) + + # FIXME this check can be removed when the minimum bitbake version has been bumped + if hasattr(bb.build, 'write_taint'): + try: + newmtime = os.path.getmtime(".config") + except OSError: + newmtime = 0 + + if newmtime > mtime: + bb.note("Configuration changed, recompile will be forced") + bb.build.write_taint('do_compile', d) } do_menuconfig[nostamp] = "1" addtask menuconfig after do_configure
Signature · 1 line
1.7.9.5
PATCH 3/8 bitbake: bitbake: ensure -f causes dependent tasks to be re-run
In reply to Scott GarmanFrom: Paul Eggleton <[email protected]>
If -f is specified, force dependent tasks to be re-run next time. This works by changing the force behaviour so that instead of deleting the task's stamp, we write a "taint" file into the stamps directory, which will alter the taskhash randomly and thus trigger the task to re-run next time we evaluate whether or not that should be done as well as influencing the taskhashes of any dependent tasks so that they are similarly re-triggered. As a bonus because we write this file as <stamp file name>.taskname.taint, the existing code which deletes the stamp files in OE's do_clean will already handle removing it.
This means you can now do the following:
bitbake somepackage
[ change the source code in the package's WORKDIR ] bitbake -c compile -f somepackage
bitbake somepackage
and the result will be that all of the tasks that depend on do_compile (do_install, do_package, etc.) will be re-run in the last step.
Note that to operate in the manner described above you need full hashing enabled (i.e. BB_SIGNATURE_HANDLER must be set to a signature handler that inherits from BasicHash). If this is not the case, -f will just delete the stamp for the specified task as it did before.
This fix is required for [YOCTO #2615] and [YOCTO #2256].
(Bitbake rev: f7b55a94226f9acd985f87946e26d01bd86a35bb)
--- bitbake/lib/bb/build.py | 18 ++++++++++++++++++ bitbake/lib/bb/cooker.py | 6 +++--- bitbake/lib/bb/runqueue.py | 12 ++++++------ bitbake/lib/bb/siggen.py | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 61 insertions(+), 9 deletions(-)
bitbake/lib/bb/build.py +18 -0
@@ -458,6 +458,24 @@ def del_stamp(task, d, file_name = None): stamp = stamp_internal(task, d, file_name) bb.utils.remove(stamp) +def write_taint(task, d, file_name = None): + """ + Creates a "taint" file which will force the specified task and its + dependents to be re-run the next time by influencing the value of its + taskhash. + (d can be a data dict or dataCache) + """ + import uuid + if file_name: + taintfn = d.stamp[file_name] + '.' + task + '.taint' + else: + taintfn = d.getVar('STAMP', True) + '.' + task + '.taint' + bb.utils.mkdirhier(os.path.dirname(taintfn)) + # The specific content of the taint file is not really important, + # we just need it to be random, so a random UUID is used + with open(taintfn, 'w') as taintf: + taintf.write(str(uuid.uuid4())) + def stampfile(taskname, d, file_name = None): """ Return the stamp for a given task
bitbake/lib/bb/cooker.py +3 -3
@@ -1066,10 +1066,10 @@ class BBCooker: self.status.rundeps[fn] = [] self.status.runrecs[fn] = [] - # Remove stamp for target if force mode active + # Invalidate task for target if force mode active if self.configuration.force: - logger.verbose("Remove stamp %s, %s", task, fn) - bb.build.del_stamp('do_%s' % task, self.status, fn) + logger.verbose("Invalidate task %s, %s", task, fn) + bb.parse.siggen.invalidate_task('do_%s' % task, self.status, fn) # Setup taskdata structure taskdata = bb.taskdata.TaskData(self.configuration.abort)
bitbake/lib/bb/runqueue.py +6 -6
@@ -705,6 +705,12 @@ class RunQueueData: continue self.runq_setscene.append(task) + # Invalidate task if force mode active + if self.cooker.configuration.force: + for (fn, target) in self.target_pairs: + logger.verbose("Invalidate task %s, %s", target, fn) + bb.parse.siggen.invalidate_task(target, self.dataCache, fn) + # Interate over the task list and call into the siggen code dealtwith = set() todeal = set(range(len(self.runq_fnid)))
@@ -731,12 +737,6 @@ class RunQueueData: deps.append(depidentifier) self.hash_deps[identifier] = deps - # Remove stamps for targets if force mode active - if self.cooker.configuration.force: - for (fn, target) in self.target_pairs: - logger.verbose("Remove stamp %s, %s", target, fn) - bb.build.del_stamp(target, self.dataCache, fn) - return len(self.runq_fnid) def dump_data(self, taskQueue):
bitbake/lib/bb/siggen.py +34 -0
@@ -50,6 +50,10 @@ class SignatureGenerator(object): def dump_sigtask(self, fn, task, stampbase, runtime): return + def invalidate_task(self, task, d, fn): + bb.build.del_stamp(task, d, fn) + + class SignatureGeneratorBasic(SignatureGenerator): """ """
@@ -148,6 +152,15 @@ class SignatureGeneratorBasic(SignatureGenerator): return False return True + def read_taint(self, fn, task, stampbase): + taint = None + try: + with open(stampbase + '.' + task + '.taint', 'r') as taintf: + taint = taintf.read() + except IOError: + pass + return taint + def get_taskhash(self, fn, task, deps, dataCache): k = fn + "." + task data = dataCache.basetaskhash[k]
@@ -161,6 +174,11 @@ class SignatureGeneratorBasic(SignatureGenerator): bb.fatal("%s is not in taskhash, caller isn't calling in dependency order?", dep) data = data + self.taskhash[dep] self.runtaskdeps[k].append(dep) + + taint = self.read_taint(fn, task, dataCache.stamp[fn]) + if taint: + data = data + taint + h = hashlib.md5(data).hexdigest() self.taskhash[k] = h #d.setVar("BB_TASKHASH_task-%s" % task, taskhash[task])
@@ -201,9 +219,14 @@ class SignatureGeneratorBasic(SignatureGenerator): for dep in data['runtaskdeps']: data['runtaskhashes'][dep] = self.taskhash[dep] + taint = self.read_taint(fn, task, stampbase) + if taint: + data['taint'] = taint + p = pickle.Pickler(file(sigfile, "wb"), -1) p.dump(data) + def dump_sigs(self, dataCache): for fn in self.taskdeps: for task in self.taskdeps[fn]:
@@ -230,6 +253,9 @@ class SignatureGeneratorBasicHash(SignatureGeneratorBasic): h = self.basehash[k] return ("%s.%s.%s.%s" % (stampbase, taskname, h, extrainfo)).rstrip('.') + def invalidate_task(self, task, d, fn): + bb.build.write_taint(task, d, fn) + def dump_this_task(outfile, d): import bb.parse fn = d.getVar("BB_FILENAME", True)
@@ -330,6 +356,11 @@ def compare_sigfiles(a, b): for dep in changed: print "Hash for dependent task %s changed from %s to %s" % (dep, a[dep], b[dep]) + a_taint = a_data.get('taint', None) + b_taint = b_data.get('taint', None) + if a_taint != b_taint: + print "Taint (by forced/invalidated task) changed from %s to %s" % (a_taint, b_taint) + def dump_sigfile(a): p1 = pickle.Unpickler(file(a, "rb")) a_data = p1.load()
@@ -354,3 +385,6 @@ def dump_sigfile(a): if 'runtaskhashes' in a_data: for dep in a_data['runtaskhashes']: print "Hash for dependent task %s is %s" % (dep, a_data['runtaskhashes'][dep]) + + if 'taint' in a_data: + print "Tainted (by forced/invalidated task): %s" % a_data['taint']
Signature · 1 line
1.7.9.5
PATCH 4/8 bitbake: lib/bb/siggen.py: log when tainting the signature of a task
In reply to Scott GarmanFrom: Paul Eggleton <[email protected]>
Log a note when applying a taint to a task signature (e.g. when using the -f or -C command line options) so that the user knows this has been done.
(Bitbake rev: 0fd960fdea83874eedb541cbc2920257e0f3fb81)
--- bitbake/lib/bb/siggen.py | 1 + 1 file changed, 1 insertion(+)
bitbake/lib/bb/siggen.py +1 -0
@@ -254,6 +254,7 @@ class SignatureGeneratorBasicHash(SignatureGeneratorBasic): return ("%s.%s.%s.%s" % (stampbase, taskname, h, extrainfo)).rstrip('.') def invalidate_task(self, task, d, fn): + bb.note("Tainting hash to force rebuild of task %s, %s" % (fn, task)) bb.build.write_taint(task, d, fn) def dump_this_task(outfile, d):
Signature · 1 line
1.7.9.5
PATCH 5/8 gnutls: Update SRC_URI to use GNU_MIRROR
In reply to Scott GarmanFrom: Darren Hart <[email protected]>
The current SRC_URI fails. Update it with the GNU_MIRROR SRC_URI from upstream commit 753b22012f10c393c191d3116b9d38ee4be6d112.
--- meta/recipes-support/gnutls/gnutls.inc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-support/gnutls/gnutls.inc +2 -2
@@ -3,7 +3,7 @@ HOMEPAGE = "http://www.gnu.org/software/gnutls/" BUGTRACKER = "https://savannah.gnu.org/support/?group=gnutls" DEPENDS = "zlib lzo libtasn1 libgcrypt (>= 1.4.2) libcap readline" -INC_PR = "r6" +INC_PR = "r7" LICENSE = "GPLv3+ & LGPLv2.1+" LICENSE_${PN} = "LGPLv2.1+"
@@ -15,7 +15,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \ file://lib/COPYING;md5=a6f89e2100d9b6cdffcea4f398e37343 \ file://libextra/COPYING;md5=d32239bcb673463ab874e80d47fae504" -SRC_URI = "ftp://ftp.gnutls.org/pub/gnutls/gnutls-${PV}.tar.bz2" +SRC_URI = "${GNU_MIRROR}/gnutls/gnutls-${PV}.tar.bz2" inherit autotools binconfig pkgconfig gettext
Signature · 1 line
1.7.9.5
PATCH 6/8 siteconfig: Clear cache before rebuilding
In reply to Scott GarmanFrom: Richard Purdie <[email protected]>
This ensures consistent build results and avoids build failures when compiler flags change for example.
(From OE-Core rev: a5ff8396cad130f809f8f8da49bb38e6f80f923c)
--- meta/classes/siteconfig.bbclass | 1 + 1 file changed, 1 insertion(+)
meta/classes/siteconfig.bbclass +1 -0
@@ -18,6 +18,7 @@ siteconfig_do_siteconfig_gencache () { >${WORKDIR}/site_config_${MACHINE}/configure.ac cd ${WORKDIR}/site_config_${MACHINE} autoconf + rm -f ${PN}_cache CONFIG_SITE="" ${EXTRASITECONFIG} ./configure ${CONFIGUREOPTS} --cache-file ${PN}_cache sed -n -e "/ac_cv_c_bigendian/p" -e "/ac_cv_sizeof_/p" \ -e "/ac_cv_type_/p" -e "/ac_cv_header_/p" -e "/ac_cv_func_/p" \
Signature · 1 line
1.7.9.5
PATCH 7/8 openjade-native: fix undefined Getopts error, use std namespace
In reply to Scott GarmanFrom: Dennis Lan <[email protected]>
Using Gentoo Linux as the build host, it fails without this patch Use Getopt::Std in place of getopts.pl.
https://bugs.gentoo.org/show_bug.cgi?id=420083
which following error:/usr/bin/perl -w ./../msggen.pl -l jstyleModule InterpreterMessages.msg /usr/bin/perl -w ./../msggen.pl -l jstyleModule DssslAppMessages.msg Undefined subroutine &main::Getopts called at ./../msggen.pl line 22. make[2]: *** [InterpreterMessages.h] Error 2
make[2]: *** Waiting for unfinished jobs....
Undefined subroutine &main::Getopts called at ./../msggen.pl line 22. make[2]: *** [DssslAppMessages.h] Error 2
(from OE-Core rev 169a89b10817b742c063fcd76721e4dbbcca6199)
--- .../openjade/openjade-1.3.2/msggen.pl.patch | 44 ++++++++++++++++++++ .../openjade/openjade-native_1.3.2.bb | 3 +- 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-devtools/openjade/openjade-1.3.2/msggen.pl.patch
meta/recipes-devtools/openjade/openjade-1.3.2/msggen.pl.patch new file +44 -0
@@ -0,0 +1,44 @@ +commit fcc5b94f118495b1a467edcda6c6f631691c3f69 +Author: Dennis Lan <[email protected]> +Date: Tue Jul 3 09:25:42 2012 +0800 + + openjade: fix undefined Getopts error, use std namespace + + Using Gentoo Linux as the build host, it fails without this patch + Use Getopt::Std in place of getopts.pl. + + Upstream-Status: Inappropriate [no upstream] + Original-Author-By: Mike Gilbert <[email protected]> + Signed-off-by: Dennis Lan <[email protected]> + +diff --git a/msggen.pl b/msggen.pl +index 0c33968..2ee3f66 100644 +--- a/msggen.pl ++++ b/msggen.pl +@@ -4,6 +4,7 @@ + # See the file COPYING for copying permission. + + use POSIX; ++use Getopt::Std; + + # Package and version. + $package = 'openjade'; +@@ -18,8 +19,7 @@ $gen_c = 0; + undef $opt_l; + undef $opt_p; + undef $opt_t; +-do 'getopts.pl'; +-&Getopts('l:p:t:'); ++getopts('l:p:t:'); + $module = $opt_l; + $pot_file = $opt_p; + +@@ -72,7 +72,7 @@ while (<DEF>) { + else { + $field[0] =~ /^[IWQXE][0-9]$/ || &error("invalid first field");; + $type[$num] = substr($field[0], 0, 1); +- $argc = int(substr($field[0], 1, 1)); ++ $argc = substr($field[0], 1, 1); + } + $nargs[$num] = $argc; + $field[1] =~ /^[a-zA-Z_][a-zA-Z0-9_]+$/ || &error("invalid tag");
meta/recipes-devtools/openjade/openjade-native_1.3.2.bb +2 -1
@@ -7,13 +7,14 @@ SECTION = "base" LICENSE = "BSD" LIC_FILES_CHKSUM = "file://COPYING;md5=641ff1e4511f0a87044ad42f87cb1045" -PR = "r4" +PR = "r5" DEPENDS = "opensp-native sgml-common-native" RDEPENDS_${PN} = "sgml-common-native" SRC_URI = "${SOURCEFORGE_MIRROR}/openjade/openjade-${PV}.tar.gz \ file://makefile.patch \ + file://msggen.pl.patch \ file://reautoconf.patch \ file://user-declared-default-constructor.patch"
Signature · 1 line
1.7.9.5
PATCH 8/8 openssl: add deprecated and unmaintained find.pl from perl-5.14 to fix perlpath.pl
In reply to Scott GarmanFrom: Martin Jansa <[email protected]>
* openembedded-core/meta/recipes-connectivity/openssl/openssl.inc *
* is using perlpath.pl:
*
* do_configure () {
* cd util
* perl perlpath.pl ${STAGING_BINDIR_NATIVE}
* ...
*
* and perlpath.pl is using find.pl:
* openssl-1.0.0i/util/perlpath.pl:
* #!/usr/local/bin/perl * # * # modify the '#!/usr/local/bin/perl' * # line in all scripts that rely on perl. * #
*
* require "find.pl";
* ...
*
* which was removed in perl-5.16.0 and marked as deprecated and
* unmaintained in 5.14 and older:
* /tmp/usr/lib/perl5/5.14.2/find.pl:
* warn "Legacy library @{[(caller(0))[6]]} will be removed from the Perl
* core distribution in the next major release. Please install it from the
* CPAN distribution Perl4::CoreLibs. It is being used at @{[(caller)[1]]},
* line @{[(caller)[2]]}.\n";
*
* # This library is deprecated and unmaintained. It is included for
* # compatibility with Perl 4 scripts which may use it, but it will be
* # removed in a future version of Perl. Please use the File::Find module
* # instead.(from OE-Core rev c09bf5d177a7ecd2045ef7e13fff4528137a9775)
--- .../openssl/openssl-1.0.0i/find.pl | 54 ++++++++++++++++++++ .../recipes-connectivity/openssl/openssl_1.0.0i.bb | 7 ++- 2 files changed, 60 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-connectivity/openssl/openssl-1.0.0i/find.pl
meta/recipes-connectivity/openssl/openssl-1.0.0i/find.pl new file +54 -0
@@ -0,0 +1,54 @@ +warn "Legacy library @{[(caller(0))[6]]} will be removed from the Perl core distribution in the next major release. Please install it from the CPAN distribution Perl4::CoreLibs. It is being used at @{[(caller)[1]]}, line @{[(caller)[2]]}.\n"; + +# This library is deprecated and unmaintained. It is included for +# compatibility with Perl 4 scripts which may use it, but it will be +# removed in a future version of Perl. Please use the File::Find module +# instead. + +# Usage: +# require "find.pl"; +# +# &find('/foo','/bar'); +# +# sub wanted { ... } +# where wanted does whatever you want. $dir contains the +# current directory name, and $_ the current filename within +# that directory. $name contains "$dir/$_". You are cd'ed +# to $dir when the function is called. The function may +# set $prune to prune the tree. +# +# For example, +# +# find / -name .nfs\* -mtime +7 -exec rm -f {} \; -o -fstype nfs -prune +# +# corresponds to this +# +# sub wanted { +# /^\.nfs.*$/ && +# (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) && +# int(-M _) > 7 && +# unlink($_) +# || +# ($nlink || (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_))) && +# $dev < 0 && +# ($prune = 1); +# } +# +# Set the variable $dont_use_nlink if you're using AFS, since AFS cheats. + +use File::Find (); + +*name = *File::Find::name; +*prune = *File::Find::prune; +*dir = *File::Find::dir; +*topdir = *File::Find::topdir; +*topdev = *File::Find::topdev; +*topino = *File::Find::topino; +*topmode = *File::Find::topmode; +*topnlink = *File::Find::topnlink; + +sub find { + &File::Find::find(\&wanted, @_); +} + +1;
meta/recipes-connectivity/openssl/openssl_1.0.0i.bb +6 -1
@@ -6,7 +6,7 @@ DEPENDS += "ocf-linux" CFLAG += "-DHAVE_CRYPTODEV -DUSE_CRYPTODEV_DIGESTS" -PR = "${INC_PR}.2" +PR = "${INC_PR}.3" LIC_FILES_CHKSUM = "file://LICENSE;md5=f9a8f968107345e0b75aa8c2ecaa7ec8"
@@ -29,6 +29,7 @@ SRC_URI += "file://configure-targets.patch \ file://debian/no-symbolic.patch \ file://debian/debian-targets.patch \ file://openssl_fix_for_x32.patch \ + file://find.pl \ " SRC_URI[md5sum] = "b4df9c11af454fd68178c85a1d5f328f"
@@ -43,3 +44,7 @@ FILES_${PN}-engines = "${libdir}/ssl/engines/*.so ${libdir}/engines" FILES_${PN}-engines-dbg = "${libdir}/ssl/engines/.debug" PARALLEL_MAKEINST = "" + +do_configure_prepend() { + cp ${WORKDIR}/find.pl ${S}/util/find.pl +}
Signature · 1 line
1.7.9.5
Richard Purdie
AppliedQuoted 5 lines · Scott Garman
Hi Richard,
This pull request correspons to the one I submitted to oe-core recently and that you've already merged, except for the two bitbake patches which we also need for the poky tree.
Merged to denzil, thanks.
Richard