Skip to contentSkip to conversation

RFC: Refactor/cleanup logging/run file generation

PATCH Applied Mark Hatle 3 participants 24 messages May 22, 2012 – May 30, 2012

lore
Share & tools
lore

Mark Hatle

author
#0 lore raw

cover RFC: Refactor/cleanup logging/run file generation

This set of commits refactors update-alternatives. The primary purpose is to cover all of the corner cases currently in oe-core (and hopefully other layers) so that recipes can all use the update-alternatives class, instead of manual update-alternatives calls in man post install/remove scripts.

The refactored update-alternatives is compatible with the older methods, but does indicate they are deprecated via a warning message. One of the key changes is that all referenced alternatives are not added to the per-file dependency information. For example, gawk sets up an alternative for 'awk'. The /bin/gawk binary now says it provides /bin/awk, so that the package management systems that use per file dependencies (i.e. RPM), will be able to find the minimum set of packages to resolve the image.

In addition, to all of that, the references have been changed from ".${PN}" to ".${BPN}" for alternatives. This will allow the multilib system to work as intended by resolving multilib binaries.

All of the work has been tested using buildhistory and building various images. Buildhistory indicated only expected changes were performed.

(To avoid spamming the list, I have decided to only send the first 3 patches, lrzsz, busybox, util-linux and shadow-native as this is a reasonable representation of the changes. Please see the referenced repository and branch below for full details on the series.)

*** BLURB HERE ***
The following changes since commit e6333825c3482a559a0c0499e17f8f48d3042ddf:

tune-mips64.inc: Add new tune file for mips64 big-endian (2012-05-20 20:24:37 -0700)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/upd-alt-2
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/upd-alt-2

Mark Hatle (30):
update-alternatives.bbclass: Refactor the implementation update-alternatives.bbclass: Change from using PN to BPN
which: use BPN for alternatives
wget: use BPN for alternatives
time: use BPN for alternatives
shadow: use BPN for alternatives
sed: use BPN for alternatives
mktemp: use BPN for alternatives
less: use BPN for alternatives
patch: use BPN for alternatives
tinylogin: use BPN for alternatives
sysvinit: use new update-alternatives
lrzsz: use new update-alternatives
openssh: use new update-alternatives
busybox: use new update-alternatives
coreutils: use new update-alternatives
dropbear: use new update-alternatives
ncurses: use new update-alternatives
psplash: use new update-alternatives
util-linux: use new update-alternatives
binutils: use new update-alternatives
tar: use new update-alternatives
opkg: use new update-alternatives
cpio: use new update-alternatives
grep: use new update-alternatives
iputils: use new update-alternatives
psmisc: use new update-alternatives
sysklogd: use new update-alternatives
kmod: use new update-alternatives
shadow-utils-native: remove unnecessary alternatives

 meta/classes/package.bbclass                       |   16 +-
 meta/classes/package_rpm.bbclass                   |    4 -
 meta/classes/update-alternatives.bbclass           |  348 ++++++++++++++++----
 meta/recipes-bsp/lrzsz/lrzsz_0.12.20.bb            |   30 +-
 meta/recipes-connectivity/openssh/openssh_6.0p1.bb |   31 +--
 meta/recipes-core/busybox/busybox.inc              |   66 ++---
 meta/recipes-core/busybox/busybox_1.19.4.bb        |    2 +-
 meta/recipes-core/coreutils/coreutils_6.9.bb       |   48 +--
 meta/recipes-core/coreutils/coreutils_8.14.bb      |   49 ++--
 meta/recipes-core/dropbear/dropbear.inc            |   12 +-
 meta/recipes-core/dropbear/dropbear_2012.55.bb     |    2 +-
 meta/recipes-core/ncurses/ncurses.inc              |   24 +-
 meta/recipes-core/psplash/psplash_git.bb           |   23 +-
 meta/recipes-core/sysvinit/sysvinit/postinst       |   17 -
 meta/recipes-core/sysvinit/sysvinit/prerm          |    9 -
 meta/recipes-core/sysvinit/sysvinit_2.88dsf.bb     |   65 +---
 meta/recipes-core/tinylogin/tinylogin_1.4.bb       |    4 +-
 meta/recipes-core/util-linux/util-linux.inc        |  212 +++----------
 meta/recipes-core/util-linux/util-linux_2.21.1.bb  |    2 +-
 meta/recipes-devtools/binutils/binutils.inc        |   18 +-
 meta/recipes-devtools/binutils/binutils_2.22.bb    |    2 +-
 meta/recipes-devtools/opkg/opkg.inc                |   13 +-
 meta/recipes-devtools/patch/patch.inc              |    4 +-
 meta/recipes-extended/cpio/cpio_2.11.bb            |    2 +-
 meta/recipes-extended/cpio/cpio_2.8.bb             |    2 +-
 meta/recipes-extended/cpio/cpio_v2.inc             |   29 +-
 meta/recipes-extended/grep/grep_2.5.1a.bb          |   25 +-
 meta/recipes-extended/grep/grep_2.9.bb             |   26 +-
 meta/recipes-extended/iputils/iputils_s20101006.bb |   34 +-
 meta/recipes-extended/less/less_444.bb             |    4 +-
 meta/recipes-extended/mktemp/mktemp_1.7.bb         |    4 +-
 meta/recipes-extended/psmisc/psmisc.inc            |   26 +--
 meta/recipes-extended/psmisc/psmisc_22.16.bb       |    2 +-
 meta/recipes-extended/sed/sed_4.1.2.bb             |    4 +-
 meta/recipes-extended/sed/sed_4.2.1.bb             |    4 +-
 .../shadow/shadow-native_4.1.4.3.bb                |   21 --
 meta/recipes-extended/shadow/shadow_4.1.4.3.bb     |    4 +-
 meta/recipes-extended/sysklogd/sysklogd.inc        |   29 +-
 meta/recipes-extended/sysklogd/sysklogd_1.5.bb     |    2 +-
 meta/recipes-extended/tar/tar.inc                  |   18 +-
 meta/recipes-extended/tar/tar_1.17.bb              |    2 +-
 meta/recipes-extended/tar/tar_1.26.bb              |    2 +-
 meta/recipes-extended/time/time.inc                |    4 +-
 meta/recipes-extended/wget/wget.inc                |    4 +-
 meta/recipes-extended/which/which_2.18.bb          |    4 +-
 meta/recipes-extended/which/which_2.20.bb          |    4 +-
 meta/recipes-kernel/kmod/kmod_git.bb               |   39 +--
 47 files changed, 603 insertions(+), 693 deletions(-)
 delete mode 100644 meta/recipes-core/sysvinit/sysvinit/postinst
 delete mode 100644 meta/recipes-core/sysvinit/sysvinit/prerm
Signature · 1 line
1.7.3.4
author
#1 lore raw

PATCH 1/30 update-alternatives.bbclass: Refactor the implementation

Refactor in order to:
* Deprecate the old interfaces, but keep them for compatibility
* Provide a new, interface -- capable of working with split packages
* Each update-alternative will not set proper "per-file" provides

Note: this adds a warning message when the older deprecated behavior is used. The older behavior has been fully tested using oe-core.

---
 meta/classes/package.bbclass             |   16 +-
 meta/classes/package_rpm.bbclass         |    4 -
 meta/classes/update-alternatives.bbclass |  344 ++++++++++++++++++++++++------
 3 files changed, 293 insertions(+), 71 deletions(-)
meta/classes/package.bbclass +10 -6
index 6fba5b6..1b776ae 100644
@@ -1156,6 +1156,15 @@ python package_do_filedeps() {
 	rpmdeps = d.expand("${RPMDEPS}")
 	r = re.compile(r'[<>=]+ +[^ ]*')
 
+	def file_translate(file):
+		ft = file.replace("@", "@at@")
+		ft = ft.replace(" ", "@space@")
+		ft = ft.replace("\t", "@tab@")
+		ft = ft.replace("[", "@openbrace@")
+		ft = ft.replace("]", "@closebrace@")
+		ft = ft.replace("_", "@underscore@")
+		return ft
+
 	# Quick routine to process the results of the rpmdeps call...
 	def process_deps(pipe, pkg, provides_files, requires_files):
 		provides = {}
@@ -1173,12 +1182,7 @@ python package_do_filedeps() {
 				continue
 
 			file = f.replace(pkgdest + "/" + pkg, "")
-			file = file.replace("@", "@at@")
-			file = file.replace(" ", "@space@")
-			file = file.replace("\t", "@tab@")
-			file = file.replace("[", "@openbrace@")
-			file = file.replace("]", "@closebrace@")
-			file = file.replace("_", "@underscore@")
+			file = file_translate(file)
 			value = line.split(":", 1)[1].strip()
 			value = r.sub(r'(\g<0>)', value)
 
meta/classes/package_rpm.bbclass +0 -4
index 2da7a8b..df2b884 100644
@@ -710,10 +710,6 @@ python write_specfile () {
 		splitrconflicts  = strip_multilib(localdata.getVar('RCONFLICTS', True), d) or ""
 		splitrobsoletes  = []
 
-		# For now we need to manually supplement RPROVIDES with any update-alternatives links
-		if pkg == d.getVar("PN", True):
-			splitrprovides = splitrprovides + " " + (d.getVar('ALTERNATIVE_LINK', True) or '') + " " + (d.getVar('ALTERNATIVE_LINKS', True) or '')
-
 		# Gather special src/first package data
 		if srcname == splitname:
 			srcrdepends    = splitrdepends
meta/classes/update-alternatives.bbclass +283 -61

368 lines not on this page · raw message

Signature · 1 line
1.7.3.4
#2 lore raw

Re: [PATCH 01/30] update-alternatives.bbclass: Refactor the implementation

Op 22 mei 2012, om 21:30 heeft Mark Hatle het volgende geschreven:

Quoted 4 lines

Refactor in order to:
* Deprecate the old interfaces, but keep them for compatibility
* Provide a new, interface -- capable of working with split packages
* Each update-alternative will not set proper "per-file" provides

'now' or 'not'?

author
#3 lore raw

Re: [PATCH 01/30] update-alternatives.bbclass: Refactor the implementation

Quoted 9 lines · Koen Kooi

Op 22 mei 2012, om 21:30 heeft Mark Hatle het volgende geschreven:

Quoted 4 lines

Refactor in order to:
* Deprecate the old interfaces, but keep them for compatibility
* Provide a new, interface -- capable of working with split packages
* Each update-alternative will not set proper "per-file" provides

'now' or 'not'?

One typo.. ;) yes it should be 'now'.

I'll fix it in the repository and branch.

--Mark

Quoted 4 lines

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

author
#4 lore raw

PATCH 2/30 update-alternatives.bbclass: Change from using PN to BPN

In reply to Mark Hatle

In order to better support multilib processing, switch from PN to BPN.

---
 meta/classes/update-alternatives.bbclass |   16 ++++++++--------
 1 files changed, 8 insertions(+), 8 deletions(-)
meta/classes/update-alternatives.bbclass +8 -8
index e235299..fce1e84 100644
@@ -42,7 +42,7 @@
 # from ALTERNATIVE_LINK_NAME.
 #
 # NOTE: If the ALTERNATIVE_LINK_NAME and ALTERNATIVE_TARGET are the same,
-# ALTERNATIVE_TARGET will have '.{PN}' appended to it.  If the file
+# ALTERNATIVE_TARGET will have '.{BPN}' appended to it.  If the file
 # referenced has not been renamed, it will also be renamed.  (This avoids
 # the need to rename alternative files in the do_install step, but still
 # supports it if necessary for some reason.)
@@ -91,8 +91,8 @@
 # listed in ALTERNATIVE_LINKS, say /path/cmd:
 #
 #     the name of the alternative would be: cmd
-#     the path of installed alternative would be: /path/cmd.${PN}
-#     ${D}/path/cmd will be renamed to ${D}/path/cmd.{PN} automatically
+#     the path of installed alternative would be: /path/cmd.${BPN}
+#     ${D}/path/cmd will be renamed to ${D}/path/cmd.{BPN} automatically
 #     priority will be the same from ALTERNATIVE_PRIORITY
 #
 # If above assumption breaks your requirement, then you still need to use
@@ -107,7 +107,7 @@ def update_alternatives_after_parse(d):
         return
 
     # The following code is deprecated, but included for compatibility with older packages
-    pn = d.getVar('PN', True)
+    pn = d.getVar('BPN', True)
 
     if d.getVar('ALTERNATIVE_LINKS') != None:
         # Convert old format to new format...
@@ -200,7 +200,7 @@ populate_packages[vardeps] += "${UPDALTVARS} ${@gen_updatealternativesvars(d)}"
 # place.
 python perform_packagecopy_append () {
 	# Check for deprecated usage...
-	pn = d.getVar('PN', True)
+	pn = d.getVar('BPN', True)
 	if d.getVar('ALTERNATIVE_LINKS', True) != None:
 		bb.warn('%s: Use of ALTERNATIVE_LINKS is deprecated, see update-alternatives.bbclass for more info.' % pn)
 
@@ -210,7 +210,7 @@ python perform_packagecopy_append () {
 	# Do actual update alternatives processing
 	pkgdest = d.getVar('PKGD', True)
 	for pkg in (d.getVar('PACKAGES', True) or "").split():
-		# If the src == dest, we know we need to rename the dest by appending ${PN}
+		# If the src == dest, we know we need to rename the dest by appending ${BPN}
 		link_rename = {}
 		for alt_name in (d.getVar('ALTERNATIVE_%s' % pkg, True) or "").split():
 			alt_link     = d.getVarFlag('ALTERNATIVE_LINK_NAME', alt_name, True)
@@ -254,7 +254,7 @@ python perform_packagecopy_append () {
 				bb.note('%s: Rename (link) %s -> %s' % (pn, alt_target, link_rename[alt_target]))
 				os.rename(src, dest)
 			else:
-				# Try to resolve the broken link to link.${PN}
+				# Try to resolve the broken link to link.${BPN}
 				link_maybe = '%s.%s' % (os.readlink(src), pn)
 				if os.path.lexists(os.path.join(os.path.dirname(src), link_maybe)):
 					# Ok, the renamed link target exists.. create a new link, and remove the original
@@ -313,7 +313,7 @@ python populate_packages_prepend () {
 }
 
 python package_do_filedeps_append () {
-	pn = d.getVar('PN', True)
+	pn = d.getVar('BPN', True)
 	pkgdest = d.getVar('PKGDEST', True)
 
 	for pkg in packages.split():
Signature · 1 line
1.7.3.4
author
#5 lore raw

PATCH 3/30 which: use BPN for alternatives

In reply to Mark Hatle
---
 meta/recipes-extended/which/which_2.18.bb |    4 ++--
 meta/recipes-extended/which/which_2.20.bb |    4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
meta/recipes-extended/which/which_2.18.bb +2 -2
index f87929b..d54e776 100644
@@ -27,10 +27,10 @@ do_configure_prepend() {
 	sed -i "s#${OLD}#${NEW}#g" `grep -rl ${OLD} ${S}`
 }
 do_install_append() {
-	mv ${D}/${bindir}/which ${D}/${bindir}/which.${PN}
+	mv ${D}/${bindir}/which ${D}/${bindir}/which.${BPN}
 }
 
 ALTERNATIVE_NAME = "which"
-ALTERNATIVE_PATH = "which.${PN}"
+ALTERNATIVE_PATH = "which.${BPN}"
 ALTERNATIVE_PRIORITY = "100"
 
meta/recipes-extended/which/which_2.20.bb +2 -2
index a4e860b..9b43943 100644
@@ -24,9 +24,9 @@ do_configure_prepend() {
 
 do_install() {
 	autotools_do_install
-	mv ${D}${bindir}/which ${D}${bindir}/which.${PN}
+	mv ${D}${bindir}/which ${D}${bindir}/which.${BPN}
 }
 
 ALTERNATIVE_NAME = "which"
-ALTERNATIVE_PATH = "which.${PN}"
+ALTERNATIVE_PATH = "which.${BPN}"
 ALTERNATIVE_PRIORITY = "100"
Signature · 1 line
1.7.3.4
author
#6 lore raw

PATCH 13/30 lrzsz: use new update-alternatives

In reply to Mark Hatle
---
 meta/recipes-bsp/lrzsz/lrzsz_0.12.20.bb |   30 +++++++++++++-----------------
 1 files changed, 13 insertions(+), 17 deletions(-)
meta/recipes-bsp/lrzsz/lrzsz_0.12.20.bb +13 -17
index 7f86157..12ac5ff 100644
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
 			file://src/lrz.c;beginline=1;endline=10;md5=5276956373ff7d8758837f6399a1045f"
 SECTION = "console/network"
 DEPENDS = ""
-PR = "r4"
+PR = "r5"
 
 SRC_URI = "http://www.ohse.de/uwe/releases/lrzsz-${PV}.tar.gz \
 	   file://autotools.patch \
@@ -27,20 +27,16 @@ do_install() {
 	install -m 0755 src/lrz src/lsz ${D}${bindir}/
 }
 
-pkg_postinst_${PN}() {
-	for util in rz rx rb; do
-		update-alternatives --install ${bindir}/$util $util lrz 100
-	done
-	for util in sz sx sb; do
-		update-alternatives --install ${bindir}/$util $util lsz 100
-	done
-}
+inherit update-alternatives
 
-pkg_postrm_${PN}() {
-	for util in rz rx rb; do
-		update-alternatives --remove $util ${bindir}/lrz
-	done
-	for util sz sx sb; do
-		update-alternatives --remove $util ${bindir}/lsz
-	done
-}
+ALTERNATIVE_PRIORITY = "100"
+
+ALTERNATIVE_${PN} = "rz rx rb sz sx sb"
+
+ALTERNATIVE_TARGET[rz] = "${bindir}/lrz"
+ALTERNATIVE_TARGET[rx] = "${bindir}/lrz"
+ALTERNATIVE_TARGET[rb] = "${bindir}/lrz"
+
+ALTERNATIVE_TARGET[sz] = "${bindir}/lsz"
+ALTERNATIVE_TARGET[sx] = "${bindir}/lsz"
+ALTERNATIVE_TARGET[sb] = "${bindir}/lsz"
Signature · 1 line
1.7.3.4
author
#7 lore raw

PATCH 15/30 busybox: use new update-alternatives

In reply to Mark Hatle
---
 meta/recipes-core/busybox/busybox.inc       |   66 +++++++++------------------
 meta/recipes-core/busybox/busybox_1.19.4.bb |    2 +-
 2 files changed, 23 insertions(+), 45 deletions(-)
meta/recipes-core/busybox/busybox.inc +22 -44
index 1613878..5b83d32 100644
@@ -201,36 +201,36 @@ do_install () {
 	install -m 0644 ${S}/busybox.links ${D}${sysconfdir}
 }
 
-python package_do_filedeps_append () {
-	# We need to load the full set of busybox provides from the /etc/busybox.links
-	# The pkg_postinst_ is what creates the actual links
+inherit update-alternatives
+
+ALTERNATIVE_PRIORITY = "50"
 
-	pkg = d.getVar('PN', True)
-	f_busybox = "/bin/busybox"
-	f_busybox_links = "/etc/busybox.links"
+ALTERNATIVE_${PN}-syslog = "syslog-init syslog-startup-conf"
 
-	requires_files = []
-	provides_files = []
+ALTERNATIVE_LINK_NAME[syslog-init] = "${sysconfdir}/init.d/syslog"
 
-	# Load/backup original set
-	filerprovides = d.getVar('FILERPROVIDES_%s_%s' % (f_busybox, pkg), True) or ""
+ALTERNATIVE_LINK_NAME[syslog-startup-conf] = "${sysconfdir}/syslog-startup.conf"
 
-	dep_pipe = os.popen('sed -e "s,^,%s/%s%s Provides: ," %s/%s%s' % (pkgdest, pkg, f_busybox, pkgdest, pkg, f_busybox_links))
+ALTERNATIVE_TARGET = "/bin/busybox"
+
+python do_package_prepend () {
+	# We need to load the full set of busybox provides from the /etc/busybox.links
+	# Use this to see the update-alternatives with the right information
 
-	process_deps(dep_pipe, pkg, provides_files, requires_files)
+	dvar = d.getVar('D', True)
+	pn = d.getVar('PN', True)
+	f = open('%s/etc/busybox.links' % (dvar), 'r')
 
-	# Add the new set
-	filerprovides += d.getVar('FILERPROVIDES_%s_%s' % (f_busybox, pkg), True) or ""
+	for alt_link_name in f:
+		alt_link_name = alt_link_name.strip()
+		alt_name = os.path.basename(alt_link_name)
 
-	# Make sure there is an entry for this item in the FILERPROVIDESFLIST...
-	filerprovidesflist = (d.getVar('FILERPROVIDESFLIST_%s' % pkg, True) or "").split()
-	for file in provides_files:
-		if file not in filerprovidesflist:
-			filerprovidesflist.append(file)
-	d.setVar('FILERPROVIDESFLIST_%s' % pkg, " ".join(filerprovidesflist))
+		# Match coreutils
+		if alt_name == '[':
+			alt_name = 'lbracket'
 
-	# Store the new provides
-	d.setVar('FILERPROVIDES_%s_%s' % (f_busybox, pkg), filerprovides)
+		d.appendVar('ALTERNATIVE_%s' % (pn), ' ' + alt_name)
+		d.setVarFlag('ALTERNATIVE_LINK_NAME', alt_name, alt_link_name)
 }
 
 pkg_postinst_${PN} () {
@@ -239,14 +239,6 @@ pkg_postinst_${PN} () {
 	# (update-alternatives have no problem replacing links later anyway)
 	test -n 2> /dev/null || alias test='busybox test'
 	if test "x$D" = "x"; then while read link; do if test ! -h "$link"; then case "$link" in /*/*/*) to="../../bin/busybox";; /bin/*) to="busybox";; /*/*) to="../bin/busybox";; esac; busybox ln -s $to $link; fi; done </etc/busybox.links; fi
-	
-	# This adds the links, remember that this has to work when building an image too, hence the $D
-	while read link; do case "$link" in /*/*/*) to="../../bin/busybox";; /bin/*) to="busybox";; /*/*) to="../bin/busybox";; esac; bn=`basename $link`; update-alternatives --install $link $bn $to 50; done <$D/etc/busybox.links
-}
-
-pkg_postinst_${PN}-syslog () {
-	update-alternatives --install ${sysconfdir}/init.d/syslog syslog-init syslog.${BPN} 50
-	update-alternatives --install ${sysconfdir}/syslog-startup.conf syslog-startup-conf syslog-startup.conf.${BPN} 50
 }
 
 pkg_prerm_${PN} () {
@@ -268,17 +260,6 @@ pkg_prerm_${PN} () {
 	ln -s /bin/busybox $tmpdir/sort
 	ln -s /bin/busybox $tmpdir/grep
 	export PATH=$PATH:$tmpdir
-
-	while read link
-	do
-		case "$link" in
-			/*/*/*) to="../../bin/busybox";;
-			/bin/*) to="busybox";;
-			/*/*) to="../bin/busybox";;
-		esac
-		bn=`basename $link`
-		sh /usr/bin/update-alternatives --remove $bn $to
-	done </etc/busybox.links
 }
 
 pkg_prerm_${PN}-syslog () {
@@ -288,7 +269,4 @@ pkg_prerm_${PN}-syslog () {
 			/etc/init.d/syslog stop
 		fi
 	fi
-
-	update-alternatives --remove syslog-init syslog.${BPN}
-	update-alternatives --remove syslog-startup-conf syslog-startup.conf.${BPN}
 }
meta/recipes-core/busybox/busybox_1.19.4.bb +1 -1
index a797ac9..0717075 100644
@@ -1,5 +1,5 @@
 require busybox.inc
-PR = "r5"
+PR = "r6"
 
 SRC_URI = "http://www.busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \
            file://B921600.patch \
Signature · 1 line
1.7.3.4
author
#8 lore raw

PATCH 20/30 util-linux: use new update-alternatives

In reply to Mark Hatle

In addition to switching to the new update-alternatives, it was discovered that the sln, shutdown, halt, reboot and related commands were never generated. Remove these from the list. (These appear to have been removed from util-linux some time ago.)

---
 meta/recipes-core/util-linux/util-linux.inc       |  212 +++++----------------
 meta/recipes-core/util-linux/util-linux_2.21.1.bb |    2 +-
 2 files changed, 45 insertions(+), 169 deletions(-)
meta/recipes-core/util-linux/util-linux.inc +44 -168
index 024b55c..688a5be 100644
@@ -43,15 +43,15 @@ EXTRA_OECONF = "--libdir=${base_libdir} --disable-use-tty-group \
 FILES_${PN}-doc += "${datadir}/getopt/getopt-*.*"
 
 FILES_util-linux-agetty = "${base_sbindir}/agetty"
-FILES_util-linux-fdisk = "${base_sbindir}/fdisk.${PN}"
+FILES_util-linux-fdisk = "${base_sbindir}/fdisk.${BPN}"
 FILES_util-linux-cfdisk = "${base_sbindir}/cfdisk"
 FILES_util-linux-sfdisk = "${sbindir}/sfdisk"
-FILES_util-linux-swaponoff = "${base_sbindir}/swapon.${PN} ${sbindir}/swapoff.${PN}"
-FILES_util-linux-losetup = "${base_sbindir}/losetup.${PN}"
-FILES_util-linux-mount = "${base_bindir}/mount.${PN}"
+FILES_util-linux-swaponoff = "${base_sbindir}/swapon.${BPN} ${sbindir}/swapoff.${BPN}"
+FILES_util-linux-losetup = "${base_sbindir}/losetup.${BPN}"
+FILES_util-linux-mount = "${base_bindir}/mount.${BPN}"
 FILES_util-linux-mcookie = "${bindir}/mcookie"
-FILES_util-linux-umount = "${base_bindir}/umount.${PN}"
-FILES_util-linux-readprofile = "${base_sbindir}/readprofile.${PN}"
+FILES_util-linux-umount = "${base_bindir}/umount.${BPN}"
+FILES_util-linux-readprofile = "${base_sbindir}/readprofile.${BPN}"
 FILES_util-linux-uuidgen = "${bindir}/uuidgen"
 
 FILES_util-linux-libblkid = "${base_libdir}/libblkid.so.*"
@@ -85,29 +85,6 @@ do_compile () {
 	oe_runmake ARCH=${TARGET_ARCH} CPU= CPUOPT= 'OPT=${CFLAGS}' 
 }
 
-checklink() {
-	local targ link value
-	targ="$1"
-	link="$2"
-	ext="$3"
-	if test -h "$link"
-	then
-		value="$(readlink "$link")"
-		if test "$value" = "$targ"
-		then
-			rm "$link"
-			ln -s "$targ"."${PN}" "$link$ext"
-			return 0
-		else
-			echo "$link: '$value' found '$targ' expected" >&2
-			return 1
-		fi
-	else
-		echo "$link: not a symbolic link" >&2
-		return 1
-	fi
-}
-
 do_install () {
 	# with ccache the timestamps on compiled files may
 	# end up earlier than on their inputs, this allows
@@ -119,7 +96,7 @@ do_install () {
 	mkdir -p ${D}${base_bindir}
 
         sbinprogs="agetty ctrlaltdel cfdisk vipw vigr"
-        sbinprogs_a="pivot_root hwclock mkswap shutdown mkfs.minix fsck.minix losetup swapon fdisk readprofile fsck blkid sln blockdev"
+        sbinprogs_a="pivot_root hwclock mkswap mkfs.minix fsck.minix losetup swapon fdisk readprofile fsck blkid blockdev"
         usrbinprogs_a="chfn chsh hexdump last logger mesg newgrp renice wall setsid chrt flock"
         binprogs_a="dmesg kill more umount mount login reset"
 
@@ -141,161 +118,60 @@ do_install () {
                 done
         fi
 
-        for p in $usrbinprogs_a; do
-                if [ -f "${D}${bindir}/$p" ]; then
-                        mv "${D}${bindir}/$p" "${D}${bindir}/$p.${PN}"
-                fi
-        done
-
-        for p in $binprogs_a; do
-                if [ -f "${D}${base_bindir}/$p" ]; then
-                        mv "${D}${base_bindir}/$p" "${D}${base_bindir}/$p.${PN}"
-                fi
-        done
-
-        for p in $sbinprogs_a; do
-                if [ -f "${D}${base_sbindir}/$p" ]; then
-                        mv "${D}${base_sbindir}/$p" "${D}${base_sbindir}/$p.${PN}"
-                fi
-        done
-	
 	install -d ${D}${sysconfdir}/default/
 	echo 'MOUNTALL="-t nonfs,nosmbfs,noncpfs"' > ${D}${sysconfdir}/default/mountall
 
-	# note ${PN} in swapon.${PN}, swapon is an alternative link that is
-	# not guaranteed to provide multiple functions, similar for shutdown
-	ln -sf swapon.${PN} ${D}${base_sbindir}/swapoff.${PN}
-	ln -sf shutdown.${PN} ${D}${base_sbindir}/reboot.${PN}
-	ln -sf shutdown.${PN} ${D}${base_sbindir}/halt.${PN}
-	ln -sf shutdown.${PN} ${D}${base_sbindir}/fastboot
-	ln -sf shutdown.${PN} ${D}${base_sbindir}/fasthalt
-
-	rm -f ${D}${sbindir}/halt
-	rm -f ${D}${sbindir}/reboot
-	rm -f ${D}${sbindir}/fastboot
-	rm -f ${D}${sbindir}/fasthalt
+	ln -sf swapon   ${D}${base_sbindir}/swapoff
 	rm -f ${D}${sbindir}/swapoff
 }
 
-pkg_postinst_${PN} () {
-	update-alternatives --install ${base_bindir}/dmesg dmesg dmesg.${PN} 100
-	update-alternatives --install ${base_bindir}/kill kill kill.${PN} 100
-	update-alternatives --install ${base_bindir}/more more more.${PN} 100
-	update-alternatives --install ${base_sbindir}/mkswap mkswap mkswap.${PN} 100
-	update-alternatives --install ${base_sbindir}/blockdev blockdev blockdev.${PN} 100
-	test -x ${base_sbindir}/pivot_root.${PN} && \
-	update-alternatives --install ${base_sbindir}/pivot_root pivot_root pivot_root.${PN} 100
-#	update-alternatives --install ${base_sbindir}/sln sln sln.${PN} 100
-	update-alternatives --install ${base_sbindir}/mkfs.minix mkfs.minix mkfs.minix.${PN} 100
-	update-alternatives --install ${bindir}/hexdump hexdump hexdump.${PN} 100
-	update-alternatives --install ${bindir}/last last last.${PN} 100
-	update-alternatives --install ${bindir}/logger logger logger.${PN} 100
-	update-alternatives --install ${bindir}/mesg mesg mesg.${PN} 100
-	update-alternatives --install ${bindir}/renice renice renice.${PN} 100
-	update-alternatives --install ${bindir}/wall wall wall.${PN} 100
-	update-alternatives --install ${bindir}/setsid setsid setsid.${PN} 100
-	update-alternatives --install ${bindir}/chrt chrt chrt.${PN} 100
-	update-alternatives --install ${bindir}/flock flock flock.${PN} 100
-	
-	# There seems to be problem, atleast on nslu2, with these, untill they are
-	# fixed the busybox ones have higher priority
-	update-alternatives --install ${base_sbindir}/hwclock hwclock hwclock.${PN} 10
-	update-alternatives --install ${base_sbindir}/shutdown shutdown shutdown.${PN} 10
-	update-alternatives --install ${base_sbindir}/reboot reboot reboot.${PN} 10
-	update-alternatives --install ${base_sbindir}/halt halt halt.${PN} 10
-}
+inherit update-alternatives
 
-pkg_prerm_${PN} () {
-	test -x ${base_sbindir}/pivot_root.${PN} && \
-	update-alternatives --remove pivot_root pivot_root.${PN}
-	update-alternatives --remove dmesg dmesg.${PN}
-	update-alternatives --remove kill kill.${PN}
-	update-alternatives --remove more more.${PN}
-	update-alternatives --remove halt halt.${PN}
-	update-alternatives --remove hwclock hwclock.${PN}
-	update-alternatives --remove mkswap mkswap.${PN}
-	update-alternatives --remove blockdev blockdev.${PN}
-	update-alternatives --remove reboot reboot.${PN}
-	update-alternatives --remove shutdown shutdown.${PN}
-#	update-alternatives --remove sln sln.${PN}
-	update-alternatives --remove mkfs.minix mkfs.minix.${PN}
-	update-alternatives --remove hexdump hexdump.${PN}
-	update-alternatives --remove last last.${PN}
-	update-alternatives --remove logger logger.${PN}
-	update-alternatives --remove mesg mesg.${PN}
-	update-alternatives --remove renice renice.${PN}
-	update-alternatives --remove wall wall.${PN}
-	update-alternatives --remove setsid setsid.${PN}
-	update-alternatives --remove chrt chrt.${PN}
-	update-alternatives --remove flock flock.${PN}
-}
+ALTERNATIVE_PRIORITY = "100"
 
-pkg_postinst_util-linux-fdisk () {
-	update-alternatives --install ${base_sbindir}/fdisk fdisk fdisk.${PN} 100
-}
+ALTERNATIVE_${PN}  = "dmesg kill more reset mkswap blockdev pivot_root"
+ALTERNATIVE_${PN} += "mkfs.minix hexdump last logger mesg renice wall"
+ALTERNATIVE_${PN} += "setsid chrt flock hwclock"
 
-pkg_prerm_util-linux-fdisk () {
-	update-alternatives --remove fdisk fdisk.${PN}
-}
+ALTERNATIVE_LINK_NAME[dmesg] = "${base_bindir}/dmesg"
+ALTERNATIVE_LINK_NAME[kill] = "${base_bindir}/kill"
+ALTERNATIVE_LINK_NAME[more] = "${base_bindir}/more"
+ALTERNATIVE_LINK_NAME[reset] = "${base_bindir}/reset"
+ALTERNATIVE_LINK_NAME[mkswap] = "${base_sbindir}/mkswap"
+ALTERNATIVE_LINK_NAME[blockdev] = "${base_sbindir}/blockdev"
+ALTERNATIVE_LINK_NAME[pivot_root] = "${base_sbindir}/pivot_root"
+ALTERNATIVE_LINK_NAME[mkfs.minix] = "${base_sbindir}/mkfs.minix"
 
-pkg_postinst_util-linux-agetty () {
-	update-alternatives --install ${base_sbindir}/getty getty agetty 100
-}
+# There seems to be problem, atleast on nslu2, with these, untill they are
+# fixed the busybox ones have higher priority
+ALTERNATIVE_PRIORITY[hwclock] = "10"
+ALTERNATIVE_LINK_NAME[hwclock] = "${base_sbindir}/hwclock"
 
-pkg_prerm_util-linux-agetty () {
-	update-alternatives --remove getty agetty
-}
+ALTERNATIVE_util-linux-fdisk = "fdisk"
+ALTERNATIVE_LINK_NAME[fdisk] = "${base_sbindir}/fdisk"
 
-pkg_postinst_util-linux-mount () {
-	update-alternatives --install ${base_bindir}/mount mount mount.${PN} 100
-}
+ALTERNATIVE_util-linux-agetty = "getty"
+ALTERNATIVE_LINK_NAME[getty] = "${base_sbindir}/getty"
+ALTERNATIVE_TARGET[getty] = "${base_sbindir}/agetty"
 
-pkg_prerm_util-linux-mount () {
-	update-alternatives --remove mount mount.${PN}
-}
+ALTERNATIVE_util-linux-mount = "mount"
+ALTERNATIVE_LINK_NAME[mount] = "${base_bindir}/mount"
 
-pkg_postinst_util-linux-umount () {
-	update-alternatives --install ${base_bindir}/umount umount umount.${PN} 100
-}
+ALTERNATIVE_util-linux-umount = "umount"
+ALTERNATIVE_LINK_NAME[umount] = "${base_bindir}/umount"
 
-pkg_prerm_util-linux-umount () {
-	update-alternatives --remove umount umount.${PN}
-}
+ALTERNATIVE_util-linux-losetup = "losetup"
+ALTERNATIVE_LINK_NAME[losetup] = "${base_sbindir}/losetup"
 
-pkg_postinst_util-linux-losetup () {
-	update-alternatives --install ${base_sbindir}/losetup losetup losetup.${PN} 100
-}
+ALTERNATIVE_util-linux-swaponoff = "swapoff swapon"
+ALTERNATIVE_LINK_NAME[swapoff] = "${base_sbindir}/swapoff"
+ALTERNATIVE_LINK_NAME[swapon] = "${base_sbindir}/swapon"
 
-pkg_prerm_util-linux-losetup () {
-	update-alternatives --remove losetup losetup.${PN}
-}
-
-pkg_postinst_util-linux-swaponoff () {
-	update-alternatives --install ${base_sbindir}/swapoff swapoff swapoff.${PN} 100
-	update-alternatives --install ${base_sbindir}/swapon swapon swapon.${PN} 100
-}
-
-pkg_prerm_util-linux-swaponoff () {
-	update-alternatives --remove swapoff swapoff.${PN}
-	update-alternatives --remove swapon swapon.${PN}
-}
-
-pkg_postinst_util-linux-fsck () {	
-	update-alternatives --install ${base_sbindir}/fsck.minix fsck.minix fsck.minix.${PN} 100
-	update-alternatives --install ${base_sbindir}/fsck fsck fsck.${PN} 100
-}
-
-pkg_prerm_util-linux-fsck () {
-	update-alternatives --remove fsck.minix fsck.minix.${PN}
-	update-alternatives --remove fsck fsck.${PN}
-}
+ALTERNATIVE_util-linux-fsck = "fsck.minix fsck"
+ALTERNATIVE_LINK_NAME[fsck.minix] = "${base_sbindir}/fsck.minix"
+ALTERNATIVE_LINK_NAME[fsck] = "${base_sbindir}/fsck"
 
-pkg_postinst_util-linux-blkid () {
-	update-alternatives --install ${base_sbindir}/blkid blkid blkid.${PN} 100
-}
-
-pkg_prerm_util-linux-blkid () {
-	update-alternatives --remove blkid blkid.${PN}
-}
+ALTERNATIVE_util-linux-blkid = "blkid"
+ALTERNATIVE_LINK_NAME[blkid] = "${base_sbindir}/blkid"
 
 BBCLASSEXTEND = "native"
meta/recipes-core/util-linux/util-linux_2.21.1.bb +1 -1
index 12faba0..8492cdf 100644
@@ -1,5 +1,5 @@
 MAJOR_VERSION = "2.21"
-PR = "r0"
+PR = "r1"
 require util-linux.inc
 
 # note that `lscpu' is under GPLv3+
Signature · 1 line
1.7.3.4
author
#9 lore raw

PATCH 30/30 shadow-utils-native: remove unnecessary alternatives

In reply to Mark Hatle
---
 .../shadow/shadow-native_4.1.4.3.bb                |   21 --------------------
 1 files changed, 0 insertions(+), 21 deletions(-)
meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb +0 -21
index 38bb922..5d9907b 100644
@@ -43,25 +43,4 @@ do_install_append() {
 	# Now we don't have a mail system. Disable mail creation for now.
 	sed -i 's:/bin/bash:/bin/sh:g' ${D}${sysconfdir}/default/useradd
 	sed -i '/^CREATE_MAIL_SPOOL/ s:^:#:' ${D}${sysconfdir}/default/useradd
-
-	install -d ${D}${sbindir} ${D}${base_sbindir} ${D}${base_bindir} 
-	for i in passwd chfn newgrp chsh ; do
-		mv ${D}${bindir}/$i ${D}${bindir}/$i.${PN}
-	done
-
-	mv ${D}${sbindir}/chpasswd ${D}${sbindir}/chpasswd.${PN}
-}
-
-pkg_postinst_${PN} () {
-	update-alternatives --install ${bindir}/passwd passwd passwd.${PN} 200
-	update-alternatives --install ${sbindir}/chpasswd chpasswd chpasswd.${PN} 200
-	update-alternatives --install ${bindir}/chfn chfn chfn.${PN} 200
-	update-alternatives --install ${bindir}/newgrp newgrp newgrp.${PN} 200
-	update-alternatives --install ${bindir}/chsh chsh chsh.${PN} 200
-}
-
-pkg_prerm_${PN} () {
-	for i in passwd chpasswd chfn newgrp chsh ; do
-		update-alternatives --remove $i $i.${PN}
-	done
 }
Signature · 1 line
1.7.3.4
author
#10 lore raw

PATCH 1/3 build.py: Add a log to capture task execution order

In reply to Mark Hatle

The new log.task_order contains an ordered list of the tasks as they were executed in any given recipe. The format of the lines is <task> <pid>: <log file>

---
 lib/bb/build.py |   16 +++++++++++++++-
 1 files changed, 15 insertions(+), 1 deletions(-)
lib/bb/build.py +15 -1
index 95f1dcf..4f06b15 100644
@@ -290,8 +290,22 @@ def _exec_task(fn, task, d, quieterr):
         bb.fatal("T variable not set, unable to build")
 
     bb.utils.mkdirhier(tempdir)
-    loglink = os.path.join(tempdir, 'log.{0}'.format(task))
+
+    # Determine the logfile to generate
     logbase = 'log.{0}.{1}'.format(task, os.getpid())
+
+    # Document the order of the tasks...
+    logorder = os.path.join(tempdir, 'log.task_order')
+    try:
+        logorderfile = file(logorder, 'a')
+    except OSError:
+        logger.exception("Opening log file '%s'", logorder)
+        pass
+    logorderfile.write('{0} ({1}): {2}\n'.format(task, os.getpid(), logbase))
+    logorderfile.close()
+
+    # Setup the courtesy link to the logfn
+    loglink = os.path.join(tempdir, 'log.{0}'.format(task))
     logfn = os.path.join(tempdir, logbase)
     if loglink:
         bb.utils.remove(loglink)
Signature · 1 line
1.7.3.4
author
#11 lore raw

PATCH 1/3 build.py: Add a log to capture task execution order

In reply to Mark Hatle

The new log.task_order contains an ordered list of the tasks as they were executed in any given recipe. The format of the lines is <task> <pid>: <log file>

---
 lib/bb/build.py |   16 +++++++++++++++-
 1 files changed, 15 insertions(+), 1 deletions(-)
lib/bb/build.py +15 -1
index 95f1dcf..4f06b15 100644
@@ -290,8 +290,22 @@ def _exec_task(fn, task, d, quieterr):
         bb.fatal("T variable not set, unable to build")
 
     bb.utils.mkdirhier(tempdir)
-    loglink = os.path.join(tempdir, 'log.{0}'.format(task))
+
+    # Determine the logfile to generate
     logbase = 'log.{0}.{1}'.format(task, os.getpid())
+
+    # Document the order of the tasks...
+    logorder = os.path.join(tempdir, 'log.task_order')
+    try:
+        logorderfile = file(logorder, 'a')
+    except OSError:
+        logger.exception("Opening log file '%s'", logorder)
+        pass
+    logorderfile.write('{0} ({1}): {2}\n'.format(task, os.getpid(), logbase))
+    logorderfile.close()
+
+    # Setup the courtesy link to the logfn
+    loglink = os.path.join(tempdir, 'log.{0}'.format(task))
     logfn = os.path.join(tempdir, logbase)
     if loglink:
         bb.utils.remove(loglink)
Signature · 1 line
1.7.3.4
author
#12 lore raw

PATCH 2/3 build.py: Add additional debug messages

In reply to Mark Hatle

We now add a debug message when entering and exiting a python or shell function. This makes it easier to inspect a log and figure out the run order from the logs.

Also minor cleanup in th exec_func_shell to match the exec_func_python variables.

---
 lib/bb/build.py |   20 +++++++++++++++-----
 1 files changed, 15 insertions(+), 5 deletions(-)
lib/bb/build.py +15 -5
index 4f06b15..2fbe120 100644
@@ -206,6 +206,8 @@ def exec_func_python(func, d, runfile, cwd=None):
             olddir = None
         os.chdir(cwd)
 
+    bb.debug(2, "Executing python function %s" % func)
+
     try:
         comp = utils.better_compile(code, func, bbfile)
         utils.better_exec(comp, {"d": d}, code, bbfile)
@@ -215,13 +217,15 @@ def exec_func_python(func, d, runfile, cwd=None):
 
         raise FuncFailed(func, None)
     finally:
+        bb.debug(2, "Python function %s finished" % func)
+
         if cwd and olddir:
             try:
                 os.chdir(olddir)
             except OSError:
                 pass
 
-def exec_func_shell(function, d, runfile, cwd=None):
+def exec_func_shell(func, d, runfile, cwd=None):
     """Execute a shell function from the metadata
 
     Note on directory behavior.  The 'dirs' varflag should contain a list
@@ -234,18 +238,18 @@ def exec_func_shell(function, d, runfile, cwd=None):
 
     with open(runfile, 'w') as script:
         script.write('#!/bin/sh -e\n')
-        data.emit_func(function, script, d)
+        data.emit_func(func, script, d)
 
         if bb.msg.loggerVerboseLogs:
             script.write("set -x\n")
         if cwd:
             script.write("cd %s\n" % cwd)
-        script.write("%s\n" % function)
+        script.write("%s\n" % func)
 
     os.chmod(runfile, 0775)
 
     cmd = runfile
-    if d.getVarFlag(function, 'fakeroot'):
+    if d.getVarFlag(func, 'fakeroot'):
         fakerootcmd = d.getVar('FAKEROOT', True)
         if fakerootcmd:
             cmd = [fakerootcmd, runfile]
@@ -255,11 +259,17 @@ def exec_func_shell(function, d, runfile, cwd=None):
     else:
         logfile = sys.stdout
 
+    bb.debug(2, "Executing shell function %s" % func)
+
     try:
         bb.process.run(cmd, shell=False, stdin=NULL, log=logfile)
+
     except bb.process.CmdError:
         logfn = d.getVar('BB_LOGFILE', True)
-        raise FuncFailed(function, logfn)
+        raise FuncFailed(func, logfn)
+
+    bb.debug(2, "Shell function %s finished" % func)
+
 
 def _task_data(fn, task, d):
     localdata = data.createCopy(d)
Signature · 1 line
1.7.3.4
#13 lore raw

Re: [PATCH 2/3] build.py: Add additional debug messages

Quoted 29 lines · Mark Hatle

We now add a debug message when entering and exiting a python or shell function. This makes it easier to inspect a log and figure out the run order from the logs.

Also minor cleanup in th exec_func_shell to match the exec_func_python variables.

---
 lib/bb/build.py |   20 +++++++++++++++-----
 1 files changed, 15 insertions(+), 5 deletions(-)
lib/bb/build.py +2 -0
index 4f06b15..2fbe120 100644
@@ -206,6 +206,8 @@ def exec_func_python(func, d, runfile, cwd=None):
             olddir = None
         os.chdir(cwd)
 
+    bb.debug(2, "Executing python function %s" % func)
+
     try:
         comp = utils.better_compile(code, func, bbfile)
         utils.better_exec(comp, {"d": d}, code, bbfile)
@@ -215,13 +217,15 @@ def exec_func_python(func, d, runfile, cwd=None):
 
         raise FuncFailed(func, None)
     finally:

lib/bb/build.py def exec_func_python(func, d, runfile, cwd=None): · line 228

Quoted 10 lines
lib/bb/build.py +3 -1
+        bb.debug(2, "Python function %s finished" % func)
+
         if cwd and olddir:
             try:
                 os.chdir(olddir)
             except OSError:
                 pass
 
-def exec_func_shell(function, d, runfile, cwd=None):
+def exec_func_shell(func, d, runfile, cwd=None):

Mixing variable renames with other code changes isn't really a good idea and confuses this patch :/.

Cheers,

Richard

author
#14 lore raw

PATCH 2/3 build.py: Add additional debug messages

In reply to Mark Hatle

We now add a debug message when entering and exiting a python or shell function. This makes it easier to inspect a log and figure out the run order from the logs.

Also minor cleanup in th exec_func_shell to match the exec_func_python variables.

---
 lib/bb/build.py |   20 +++++++++++++++-----
 1 files changed, 15 insertions(+), 5 deletions(-)
lib/bb/build.py +15 -5
index 4f06b15..2fbe120 100644
@@ -206,6 +206,8 @@ def exec_func_python(func, d, runfile, cwd=None):
             olddir = None
         os.chdir(cwd)
 
+    bb.debug(2, "Executing python function %s" % func)
+
     try:
         comp = utils.better_compile(code, func, bbfile)
         utils.better_exec(comp, {"d": d}, code, bbfile)
@@ -215,13 +217,15 @@ def exec_func_python(func, d, runfile, cwd=None):
 
         raise FuncFailed(func, None)
     finally:
+        bb.debug(2, "Python function %s finished" % func)
+
         if cwd and olddir:
             try:
                 os.chdir(olddir)
             except OSError:
                 pass
 
-def exec_func_shell(function, d, runfile, cwd=None):
+def exec_func_shell(func, d, runfile, cwd=None):
     """Execute a shell function from the metadata
 
     Note on directory behavior.  The 'dirs' varflag should contain a list
@@ -234,18 +238,18 @@ def exec_func_shell(function, d, runfile, cwd=None):
 
     with open(runfile, 'w') as script:
         script.write('#!/bin/sh -e\n')
-        data.emit_func(function, script, d)
+        data.emit_func(func, script, d)
 
         if bb.msg.loggerVerboseLogs:
             script.write("set -x\n")
         if cwd:
             script.write("cd %s\n" % cwd)
-        script.write("%s\n" % function)
+        script.write("%s\n" % func)
 
     os.chmod(runfile, 0775)
 
     cmd = runfile
-    if d.getVarFlag(function, 'fakeroot'):
+    if d.getVarFlag(func, 'fakeroot'):
         fakerootcmd = d.getVar('FAKEROOT', True)
         if fakerootcmd:
             cmd = [fakerootcmd, runfile]
@@ -255,11 +259,17 @@ def exec_func_shell(function, d, runfile, cwd=None):
     else:
         logfile = sys.stdout
 
+    bb.debug(2, "Executing shell function %s" % func)
+
     try:
         bb.process.run(cmd, shell=False, stdin=NULL, log=logfile)
+
     except bb.process.CmdError:
         logfn = d.getVar('BB_LOGFILE', True)
-        raise FuncFailed(function, logfn)
+        raise FuncFailed(func, logfn)
+
+    bb.debug(2, "Shell function %s finished" % func)
+
 
 def _task_data(fn, task, d):
     localdata = data.createCopy(d)
Signature · 1 line
1.7.3.4
author
#15 lore raw

PATCH 3/3 build.py: Add support for log and run filename changes

In reply to Mark Hatle

The format of the log file and run file are now selectable using BB_LOGFMT and BB_RUNFMT, respectively.

The following values may be used:

{task} - task name
{taskfunc} - task.func or func, if task==func {func} - function name, only available in BB_RUNFMT {pid} - pid

The log/run files may be placed into a subdirectory that is relative to T.

Default BB_LOGFMT is: log.{task}.{pid}
Default BB_RUNFMT is: run.{func}.{pid}

---
 lib/bb/build.py |   20 +++++++++++++++++---
 1 files changed, 17 insertions(+), 3 deletions(-)
lib/bb/build.py +17 -3
index 2fbe120..e867894 100644
@@ -174,8 +174,19 @@ def exec_func(func, d, dirs = None):
         lockfiles = None
 
     tempdir = data.getVar('T', d, 1)
-    bb.utils.mkdirhier(tempdir)
-    runfile = os.path.join(tempdir, 'run.{0}.{1}'.format(func, os.getpid()))
+
+    # or func allows items to be executed outside of the normal
+    # task set, such as buildhistory
+    task = data.getVar('BB_RUNTASK', d, 1) or func
+    if task == func:
+        taskfunc = task
+    else:
+        taskfunc = "%s.%s" % (task, func)
+
+    runfmt = data.getVar('BB_RUNFMT', d, 1) or "run.{func}.{pid}"
+    runfn = runfmt.format(taskfunc=taskfunc, task=task, func=func, pid=os.getpid())
+    runfile = os.path.join(tempdir, runfn)
+    bb.utils.mkdirhier(os.path.dirname(runfile))
 
     with bb.utils.fileslocked(lockfiles):
         if ispython:
@@ -302,7 +313,8 @@ def _exec_task(fn, task, d, quieterr):
     bb.utils.mkdirhier(tempdir)
 
     # Determine the logfile to generate
-    logbase = 'log.{0}.{1}'.format(task, os.getpid())
+    logfmt = localdata.getVar('BB_LOGFMT', True) or 'log.{task}.{pid}'
+    logbase = logfmt.format(task=task, pid=os.getpid())
 
     # Document the order of the tasks...
     logorder = os.path.join(tempdir, 'log.task_order')
@@ -338,6 +350,7 @@ def _exec_task(fn, task, d, quieterr):
     # Handle logfiles
     si = file('/dev/null', 'r')
     try:
+        bb.utils.mkdirhier(os.path.dirname(logfn))
         logfile = file(logfn, 'w')
     except OSError:
         logger.exception("Opening log file '%s'", logfn)
@@ -364,6 +377,7 @@ def _exec_task(fn, task, d, quieterr):
     bblogger.addHandler(errchk)
 
     localdata.setVar('BB_LOGFILE', logfn)
+    localdata.setVar('BB_RUNTASK', task)
 
     event.fire(TaskStarted(task, localdata), localdata)
     try:
Signature · 1 line
1.7.3.4
author
#16 lore raw

PATCH 3/3 build.py: Add support for log and run filename changes

In reply to Mark Hatle

The format of the log file and run file are now selectable using BB_LOGFMT and BB_RUNFMT, respectively.

The following values may be used:

{task} - task name
{taskfunc} - task.func or func, if task==func {func} - function name, only available in BB_RUNFMT {pid} - pid

The log/run files may be placed into a subdirectory that is relative to T.

Default BB_LOGFMT is: log.{task}.{pid}
Default BB_RUNFMT is: run.{func}.{pid}

---
 lib/bb/build.py |   20 +++++++++++++++++---
 1 files changed, 17 insertions(+), 3 deletions(-)
lib/bb/build.py +17 -3
index 2fbe120..e867894 100644
@@ -174,8 +174,19 @@ def exec_func(func, d, dirs = None):
         lockfiles = None
 
     tempdir = data.getVar('T', d, 1)
-    bb.utils.mkdirhier(tempdir)
-    runfile = os.path.join(tempdir, 'run.{0}.{1}'.format(func, os.getpid()))
+
+    # or func allows items to be executed outside of the normal
+    # task set, such as buildhistory
+    task = data.getVar('BB_RUNTASK', d, 1) or func
+    if task == func:
+        taskfunc = task
+    else:
+        taskfunc = "%s.%s" % (task, func)
+
+    runfmt = data.getVar('BB_RUNFMT', d, 1) or "run.{func}.{pid}"
+    runfn = runfmt.format(taskfunc=taskfunc, task=task, func=func, pid=os.getpid())
+    runfile = os.path.join(tempdir, runfn)
+    bb.utils.mkdirhier(os.path.dirname(runfile))
 
     with bb.utils.fileslocked(lockfiles):
         if ispython:
@@ -302,7 +313,8 @@ def _exec_task(fn, task, d, quieterr):
     bb.utils.mkdirhier(tempdir)
 
     # Determine the logfile to generate
-    logbase = 'log.{0}.{1}'.format(task, os.getpid())
+    logfmt = localdata.getVar('BB_LOGFMT', True) or 'log.{task}.{pid}'
+    logbase = logfmt.format(task=task, pid=os.getpid())
 
     # Document the order of the tasks...
     logorder = os.path.join(tempdir, 'log.task_order')
@@ -338,6 +350,7 @@ def _exec_task(fn, task, d, quieterr):
     # Handle logfiles
     si = file('/dev/null', 'r')
     try:
+        bb.utils.mkdirhier(os.path.dirname(logfn))
         logfile = file(logfn, 'w')
     except OSError:
         logger.exception("Opening log file '%s'", logfn)
@@ -364,6 +377,7 @@ def _exec_task(fn, task, d, quieterr):
     bblogger.addHandler(errchk)
 
     localdata.setVar('BB_LOGFILE', logfn)
+    localdata.setVar('BB_RUNTASK', task)
 
     event.fire(TaskStarted(task, localdata), localdata)
     try:
Signature · 1 line
1.7.3.4
author
#17 lore raw

PATCH 1/4 build.py: Add a log to capture task execution order

In reply to Mark Hatle

The new log.task_order contains an ordered list of the tasks as they were executed in any given recipe. The format of the lines is <task> <pid>: <log file>

---
 lib/bb/build.py |   16 +++++++++++++++-
 1 files changed, 15 insertions(+), 1 deletions(-)
lib/bb/build.py +15 -1
index 95f1dcf..4f06b15 100644
@@ -290,8 +290,22 @@ def _exec_task(fn, task, d, quieterr):
         bb.fatal("T variable not set, unable to build")
 
     bb.utils.mkdirhier(tempdir)
-    loglink = os.path.join(tempdir, 'log.{0}'.format(task))
+
+    # Determine the logfile to generate
     logbase = 'log.{0}.{1}'.format(task, os.getpid())
+
+    # Document the order of the tasks...
+    logorder = os.path.join(tempdir, 'log.task_order')
+    try:
+        logorderfile = file(logorder, 'a')
+    except OSError:
+        logger.exception("Opening log file '%s'", logorder)
+        pass
+    logorderfile.write('{0} ({1}): {2}\n'.format(task, os.getpid(), logbase))
+    logorderfile.close()
+
+    # Setup the courtesy link to the logfn
+    loglink = os.path.join(tempdir, 'log.{0}'.format(task))
     logfn = os.path.join(tempdir, logbase)
     if loglink:
         bb.utils.remove(loglink)
Signature · 1 line
1.7.3.4
#18 lore raw

Re: [PATCH 1/4] build.py: Add a log to capture task execution order

Quoted 8 lines · Mark Hatle

The new log.task_order contains an ordered list of the tasks as they were executed in any given recipe. The format of the lines is <task> <pid>: <log file>

---
 lib/bb/build.py |   16 +++++++++++++++-
 1 files changed, 15 insertions(+), 1 deletions(-)

Merged to master, thanks.

Richard

author
#19 lore raw

PATCH 2/4 build.py: Cleanup exec_func_shell

In reply to Mark Hatle

exec_func_python and exec_func_shell are similar, but variable usage has diverged sync the two up. Since exec_func_python is first use that as the guide for the later exec_func_shell variable naming.

---
 lib/bb/build.py |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)
lib/bb/build.py +5 -5
index 4f06b15..9f2c6a7 100644
@@ -221,7 +221,7 @@ def exec_func_python(func, d, runfile, cwd=None):
             except OSError:
                 pass
 
-def exec_func_shell(function, d, runfile, cwd=None):
+def exec_func_shell(func, d, runfile, cwd=None):
     """Execute a shell function from the metadata
 
     Note on directory behavior.  The 'dirs' varflag should contain a list
@@ -234,18 +234,18 @@ def exec_func_shell(function, d, runfile, cwd=None):
 
     with open(runfile, 'w') as script:
         script.write('#!/bin/sh -e\n')
-        data.emit_func(function, script, d)
+        data.emit_func(func, script, d)
 
         if bb.msg.loggerVerboseLogs:
             script.write("set -x\n")
         if cwd:
             script.write("cd %s\n" % cwd)
-        script.write("%s\n" % function)
+        script.write("%s\n" % func)
 
     os.chmod(runfile, 0775)
 
     cmd = runfile
-    if d.getVarFlag(function, 'fakeroot'):
+    if d.getVarFlag(func, 'fakeroot'):
         fakerootcmd = d.getVar('FAKEROOT', True)
         if fakerootcmd:
             cmd = [fakerootcmd, runfile]
@@ -259,7 +259,7 @@ def exec_func_shell(function, d, runfile, cwd=None):
         bb.process.run(cmd, shell=False, stdin=NULL, log=logfile)
     except bb.process.CmdError:
         logfn = d.getVar('BB_LOGFILE', True)
-        raise FuncFailed(function, logfn)
+        raise FuncFailed(func, logfn)
 
 def _task_data(fn, task, d):
     localdata = data.createCopy(d)
Signature · 1 line
1.7.3.4
author
#20 lore raw

PATCH 3/4 build.py: Add additional debug messages

In reply to Mark Hatle

We now add a debug message when entering and exiting a python or shell function. This makes it easier to inspect a log and figure out the run order from the logs.

---
 lib/bb/build.py |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
lib/bb/build.py +8 -0
index 9f2c6a7..363aced 100644
@@ -206,6 +206,8 @@ def exec_func_python(func, d, runfile, cwd=None):
             olddir = None
         os.chdir(cwd)
 
+    bb.debug(2, "Executing python function %s" % func)
+
     try:
         comp = utils.better_compile(code, func, bbfile)
         utils.better_exec(comp, {"d": d}, code, bbfile)
@@ -215,6 +217,8 @@ def exec_func_python(func, d, runfile, cwd=None):
 
         raise FuncFailed(func, None)
     finally:
+        bb.debug(2, "Python function %s finished" % func)
+
         if cwd and olddir:
             try:
                 os.chdir(olddir)
@@ -255,12 +259,16 @@ def exec_func_shell(func, d, runfile, cwd=None):
     else:
         logfile = sys.stdout
 
+    bb.debug(2, "Executing shell function %s" % func)
+
     try:
         bb.process.run(cmd, shell=False, stdin=NULL, log=logfile)
     except bb.process.CmdError:
         logfn = d.getVar('BB_LOGFILE', True)
         raise FuncFailed(func, logfn)
 
+    bb.debug(2, "Shell function %s finished" % func)
+
 def _task_data(fn, task, d):
     localdata = data.createCopy(d)
     localdata.setVar('BB_FILENAME', fn)
Signature · 1 line
1.7.3.4
author
#21 lore raw

PATCH 4/4 build.py: Add support for log and run filename changes

In reply to Mark Hatle

The format of the log file and run file are now selectable using BB_LOGFMT and BB_RUNFMT, respectively.

The following values may be used:

{task} - task name
{taskfunc} - task.func or func, if task==func {func} - function name, only available in BB_RUNFMT {pid} - pid

The log/run files may be placed into a subdirectory that is relative to T.

Default BB_LOGFMT is: log.{task}.{pid}
Default BB_RUNFMT is: run.{func}.{pid}

---
 lib/bb/build.py |   20 +++++++++++++++++---
 1 files changed, 17 insertions(+), 3 deletions(-)
lib/bb/build.py +17 -3
index 363aced..a9ba02d 100644
@@ -174,8 +174,19 @@ def exec_func(func, d, dirs = None):
         lockfiles = None
 
     tempdir = data.getVar('T', d, 1)
-    bb.utils.mkdirhier(tempdir)
-    runfile = os.path.join(tempdir, 'run.{0}.{1}'.format(func, os.getpid()))
+
+    # or func allows items to be executed outside of the normal
+    # task set, such as buildhistory
+    task = data.getVar('BB_RUNTASK', d, 1) or func
+    if task == func:
+        taskfunc = task
+    else:
+        taskfunc = "%s.%s" % (task, func)
+
+    runfmt = data.getVar('BB_RUNFMT', d, 1) or "run.{func}.{pid}"
+    runfn = runfmt.format(taskfunc=taskfunc, task=task, func=func, pid=os.getpid())
+    runfile = os.path.join(tempdir, runfn)
+    bb.utils.mkdirhier(os.path.dirname(runfile))
 
     with bb.utils.fileslocked(lockfiles):
         if ispython:
@@ -300,7 +311,8 @@ def _exec_task(fn, task, d, quieterr):
     bb.utils.mkdirhier(tempdir)
 
     # Determine the logfile to generate
-    logbase = 'log.{0}.{1}'.format(task, os.getpid())
+    logfmt = localdata.getVar('BB_LOGFMT', True) or 'log.{task}.{pid}'
+    logbase = logfmt.format(task=task, pid=os.getpid())
 
     # Document the order of the tasks...
     logorder = os.path.join(tempdir, 'log.task_order')
@@ -336,6 +348,7 @@ def _exec_task(fn, task, d, quieterr):
     # Handle logfiles
     si = file('/dev/null', 'r')
     try:
+        bb.utils.mkdirhier(os.path.dirname(logfn))
         logfile = file(logfn, 'w')
     except OSError:
         logger.exception("Opening log file '%s'", logfn)
@@ -362,6 +375,7 @@ def _exec_task(fn, task, d, quieterr):
     bblogger.addHandler(errchk)
 
     localdata.setVar('BB_LOGFILE', logfn)
+    localdata.setVar('BB_RUNTASK', task)
 
     event.fire(TaskStarted(task, localdata), localdata)
     try:
Signature · 1 line
1.7.3.4
#22 lore raw

Re: [PATCH 00/30] Refactor update-alternatives

In reply to Mark Hatle
Quoted 59 lines · Mark Hatle

This set of commits refactors update-alternatives. The primary purpose is to cover all of the corner cases currently in oe-core (and hopefully other layers) so that recipes can all use the update-alternatives class, instead of manual update-alternatives calls in man post install/remove scripts.

The refactored update-alternatives is compatible with the older methods, but does indicate they are deprecated via a warning message. One of the key changes is that all referenced alternatives are not added to the per-file dependency information. For example, gawk sets up an alternative for 'awk'. The /bin/gawk binary now says it provides /bin/awk, so that the package management systems that use per file dependencies (i.e. RPM), will be able to find the minimum set of packages to resolve the image.

In addition, to all of that, the references have been changed from ".${PN}" to ".${BPN}" for alternatives. This will allow the multilib system to work as intended by resolving multilib binaries.

All of the work has been tested using buildhistory and building various images. Buildhistory indicated only expected changes were performed.

(To avoid spamming the list, I have decided to only send the first 3 patches, lrzsz, busybox, util-linux and shadow-native as this is a reasonable representation of the changes. Please see the referenced repository and branch below for full details on the series.)

*** BLURB HERE ***
The following changes since commit e6333825c3482a559a0c0499e17f8f48d3042ddf:

tune-mips64.inc: Add new tune file for mips64 big-endian (2012-05-20 20:24:37 -0700)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/upd-alt-2
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/upd-alt-2

Mark Hatle (30):
update-alternatives.bbclass: Refactor the implementation update-alternatives.bbclass: Change from using PN to BPN
which: use BPN for alternatives
wget: use BPN for alternatives
time: use BPN for alternatives
shadow: use BPN for alternatives
sed: use BPN for alternatives
mktemp: use BPN for alternatives
less: use BPN for alternatives
patch: use BPN for alternatives
tinylogin: use BPN for alternatives
sysvinit: use new update-alternatives
lrzsz: use new update-alternatives
openssh: use new update-alternatives
busybox: use new update-alternatives
coreutils: use new update-alternatives
dropbear: use new update-alternatives
ncurses: use new update-alternatives
psplash: use new update-alternatives
util-linux: use new update-alternatives
binutils: use new update-alternatives
tar: use new update-alternatives

Quoted 8 lines

opkg: use new update-alternatives
cpio: use new update-alternatives
grep: use new update-alternatives
iputils: use new update-alternatives
psmisc: use new update-alternatives
sysklogd: use new update-alternatives
kmod: use new update-alternatives
shadow-utils-native: remove unnecessary alternatives

Merged to master, thanks.

I've noticed a few issues and this obviously introduces a lot of warning messages. I've posted a couple of fixes but we need a plan to get the warnings resolved...

Cheers,

Richard

author
#23 lore raw

Re: [PATCH 00/30] Refactor update-alternatives

Quoted 69 lines · Richard Purdie
Quoted 59 lines · Mark Hatle

This set of commits refactors update-alternatives. The primary purpose is to cover all of the corner cases currently in oe-core (and hopefully other layers) so that recipes can all use the update-alternatives class, instead of manual update-alternatives calls in man post install/remove scripts.

The refactored update-alternatives is compatible with the older methods, but does indicate they are deprecated via a warning message. One of the key changes is that all referenced alternatives are not added to the per-file dependency information. For example, gawk sets up an alternative for 'awk'. The /bin/gawk binary now says it provides /bin/awk, so that the package management systems that use per file dependencies (i.e. RPM), will be able to find the minimum set of packages to resolve the image.

In addition, to all of that, the references have been changed from ".${PN}" to ".${BPN}" for alternatives. This will allow the multilib system to work as intended by resolving multilib binaries.

All of the work has been tested using buildhistory and building various images. Buildhistory indicated only expected changes were performed.

(To avoid spamming the list, I have decided to only send the first 3 patches, lrzsz, busybox, util-linux and shadow-native as this is a reasonable representation of the changes. Please see the referenced repository and branch below for full details on the series.)

*** BLURB HERE ***
The following changes since commit e6333825c3482a559a0c0499e17f8f48d3042ddf:

   tune-mips64.inc: Add new tune file for mips64 big-endian (2012-05-20 20:24:37 -0700)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/upd-alt-2
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/upd-alt-2

Mark Hatle (30):
update-alternatives.bbclass: Refactor the implementation update-alternatives.bbclass: Change from using PN to BPN
which: use BPN for alternatives
wget: use BPN for alternatives
time: use BPN for alternatives
shadow: use BPN for alternatives
sed: use BPN for alternatives
mktemp: use BPN for alternatives
less: use BPN for alternatives
patch: use BPN for alternatives
tinylogin: use BPN for alternatives
sysvinit: use new update-alternatives
lrzsz: use new update-alternatives
openssh: use new update-alternatives
busybox: use new update-alternatives
coreutils: use new update-alternatives
dropbear: use new update-alternatives
ncurses: use new update-alternatives
psplash: use new update-alternatives
util-linux: use new update-alternatives
binutils: use new update-alternatives
tar: use new update-alternatives

Quoted 8 lines

opkg: use new update-alternatives
cpio: use new update-alternatives
grep: use new update-alternatives
iputils: use new update-alternatives
psmisc: use new update-alternatives
sysklogd: use new update-alternatives
kmod: use new update-alternatives
shadow-utils-native: remove unnecessary alternatives

Quoted 5 lines

Merged to master, thanks.

I've noticed a few issues and this obviously introduces a lot of warning messages. I've posted a couple of fixes but we need a plan to get the warnings resolved...

I can work through the warnings fairly quickly. I left them there intentionally. I wanted to proved the deprecated functionality was complete and worked as expected. (I forgot to note that in the original submission.)

If nobody else is working on revising the other items I'll start working through them.

--Mark

Quoted 10 lines

Cheers,

Richard

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core