Skip to contentSkip to conversation

kernel-yocto: consolidated pull request

PATCH New Bruce Ashfield 1 participant 6 messages Feb 1, 2013

lore
Share & tools
lore
#0 lore raw

cover kernel-yocto: consolidated pull request

Richard/Saul,

This pull request is a collection of bug fixes and feature additions for the 1.4 release. Documentation updates are underway for the new features, but they've soaked long enough here, so I wanted to get them out while there's lots of time to address any issues.

[PATCH 1/5] guilt: remove GIT_EXEC_BASE export

This fixes a bug that has been reported off and on for nearly a year now. I finally saw it on my local machine and was able to track down the cause .. and export of GIT_EXEC_BASE that is no longer needed. Although I may get rid of guilt shortly, it's still worth fixing this now.

[PATCH 2/5] kernel-yocto: make configuration audit details debug only

This addresses [YOCTO #3415], and simply makes the full kernel configuration audit slightly less visible, but yet still available.

[PATCH 3/5] kernel-yocto: fix .scc and .cfg matching

I ran into this while implementing a new feature, some local SRC_URIs were matching fragments incorrectly. With this fix I'm back to only getting .scc and .cfg files.

[PATCH 4/5] kernel-yocto: allow multiple / shared kernel feature directories

This is the most interesting addition of this series, and it's the start of solving the problem of having multiple people developing disjoint sets of kernel configuration blocks, or kernel features.

To promote the reuse and sharing of configuration fragments this change allows any kernel-yocto based recipe to have multiple alternate git repositories which provide kernel feature directory trees listed on the SRC_URI.

These feature directories are in addition to any in-tree kernel meta data branches that may be available (described via the KMETA variable in linux-yocto recipes).

Features found within these directories can be used from recipes via the KERNEL_FEATURES variable. Features found within a feature directory are free to include any other features that are available in any directories. In both cases the path to a feature description (a .scc file) is relative to the root of a given feature directory (which is how existing .scc files work)

The search order for features is determined by the order that repositories appear on the SRC_URI.

Normal SRC_URI rules apply to any repository that is added as a kernel feature container. A SRCREV must be supplied and it must be unpacked to a unique directory, which is controlled via the "destsuffic" url parameter.

In addition to these standard requirements, any kernel feature repository reference should identify itself via the "type=kmeta" url parameter. If type=kmeta is not supplied, the repository will not be processed for kernel features.

As an example, the following in a linux-yocto bbappend makes two additional feature directories available to KERNEL_FEATURES and fragments.

    SRC_URI += "git://git.yoctoproject.org/yocto-kernel-cache;protocol=git;branch=master;type=kmeta;name=feat1;destsuffix=kernel-cache/"
    SRC_URI += "git://${KSRC_linux_yocto_3_4};protocol=file;branch=meta;name=feat2;type=kmeta;destsuffix=kernel-features-experimental/"

    SRCREV_feat1 = "${AUTOREV}"
    SRCREV_feat2 = "${AUTOREV}"

[PATCH 5/5] kernel-yocto/linux-yocto-custom: support low bandwidth options

To support configurations where active development is not being done within the oe/bitbake build environment and restricted bandwidth situations, this commit allows the SRC_URI to point to a kernel tgz instead of a full git repository.

Outside of the upstream tgz instead of a kernel git repository, the restrictions, config and patch process is the same as any linux-yocto-custom recipe.

An example linux-yocto-custom based recipe would have a configuration like this to build the 3.7 kernel, using an externally supplied config, from the 3.7 tgz:

    SRC_URI = "http://kernel.org/pub/linux/kernel/v3.0/linux-3.7.tar.bz2"
    PV = "3.7"
    S = "${WORKDIR}/linux-3.7"
    SRC_URI[md5sum] = "5323f3faadd051e83af605a63be5ea2e"
    SRC_URI[sha256sum] = "dc08d87a579fe2918362e6666e503a95a76296419195cb499aa9dd4dbe171a9e"

[YOCTO #2686]

Cheers,

Bruce

The following changes since commit 828b1143477da079956a5541f8cc66c07ab6a3cc:

multilib: skip packages that provide virtual/kernel (2013-01-31 12:39:47 -0500)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib zedd/kernel-dev
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (5):
guilt: remove GIT_EXEC_BASE export
kernel-yocto: make configuration audit details debug only
kernel-yocto: fix .scc and .cfg matching
kernel-yocto: allow multiple / shared kernel feature directories kernel-yocto/linux-yocto-custom: support low bandwidth options

 meta/classes/kernel-yocto.bbclass                  |   77 +++++++++++++++-----
 meta/recipes-devtools/guilt/guilt-native_0.33.bb   |    3 +-
 .../kern-tools/kern-tools-native_git.bb            |    2 +-
 3 files changed, 62 insertions(+), 20 deletions(-)
Signature · 1 line
1.7.10.4
#1 lore raw

PATCH 1/5 guilt: remove GIT_EXEC_BASE export

git-core provides the following routines (among others): git-sh-setup, cd_to_toplevel, die, and more. But it is not always in the same location relative to the guilt binary if git is not part of a 'host tools' sysroot. Modern git versions don't need this, so commenting it out (until it breaks again) is the solution to the problem of it not existing, and the library routines will sourced and provided by git itself.

Since bitbake.conf has git-native as ASSUME_PROVIDED, this means that when the system git binary is used, errors such as this can be seen in failed patch logs:

| /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/guilt: line 29: /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/../libexec/git-core/git-sh-setup: No such file or directory
| /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/guilt: line 717: cd_to_toplevel: command not found
| error: patch failed: Makefile:2
| error: Makefile: patch does not apply
| /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/guilt-push: line 137: die: command not found
| [ERROR] unable to complete push
| pending patches are:
| /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/guilt: line 29: /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/../libexec/git-core/git-sh-setup: No such file or directory
| /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/guilt: line 29: /opt/poky/build/tmp/sysroots/x86_64-linux/usr/bin/../libexec/git-core/git-sh-setup: No such file or directory
| links/linux-yocto-custom/0001-linux-version-tweak.patch
| ERROR. could not update git tree

With the export removed, we have a normal "clean" failure message when the sysroot or system git is used.

---
 meta/recipes-devtools/guilt/guilt-native_0.33.bb |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
meta/recipes-devtools/guilt/guilt-native_0.33.bb +1 -2
index 6652ab6..d4dd616 100644
@@ -4,7 +4,7 @@ LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b6f3400dc1a01cebafe8a52b3f344135"
 
 PV = "0.33"
-PR = "r3"
+PR = "r4"
 
 inherit native
 
@@ -18,7 +18,6 @@ SRC_URI = "http://ftp.de.debian.org/debian/pool/main/g/guilt/guilt_${PV}.orig.ta
 	   file://guilt-push-no-series.patch \
 	   file://make_git_commands_conditional.patch \
 	   file://improve_auto_header_gen.patch \
-	   file://guilt-set-git_exec_path.patch \
 	   file://guilt-bash.patch \
 	   file://guilt-update-supported-git-versions-to-1.8.x.patch \
 	   file://optional_head_check.patch"
Signature · 1 line
1.7.10.4
#2 lore raw

PATCH 2/5 kernel-yocto: make configuration audit details debug only

In reply to Bruce Ashfield

The details of the kernel configuration audit are typically a debug action, so should be moved to bb.debug(). But in order to maintain visibility of the results, a reference to the log file is provided in the standard message.

---
 meta/classes/kernel-yocto.bbclass |    8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
meta/classes/kernel-yocto.bbclass +6 -2
index aaee441..b336e43 100644
@@ -207,7 +207,7 @@ do_kernel_configme() {
 python do_kernel_configcheck() {
     import re, string, sys, commands
 
-    bb.plain("NOTE: validating kernel configuration")
+    bb.plain("NOTE: validating kernel config, see log.do_kernel_configcheck for details")
 
     # if KMETA isn't set globally by a recipe using this routine, we need to
     # set the default to 'meta'. Otherwise, kconf_check is not passed a valid
@@ -218,7 +218,11 @@ python do_kernel_configcheck() {
     cmd = d.expand("cd ${S}; kconf_check -config- %s/meta-series ${S} ${B}" % kmeta)
     ret, result = commands.getstatusoutput("%s%s" % (pathprefix, cmd))
 
-    bb.plain( "%s" % result )
+    config_check_visibility = d.getVar( "KCONF_AUDIT_LEVEL", True ) or 1
+    if config_check_visibility == 1:
+        bb.debug( 1, "%s" % result )
+    else:
+        bb.note( "%s" % result )
 }
 
 # Ensure that the branches (BSP and meta) are on the locations specified by
Signature · 1 line
1.7.10.4
#3 lore raw

PATCH 3/5 kernel-yocto: fix .scc and .cfg matching

In reply to Bruce Ashfield

SRC_URIs that contained git repositories or other constructs that resulted in an extension of "." or a substring of "scc" or "cfg" were matching the tests for patches and configs. This was due to a python tuple being used instead of an array. Switching to an array makes the match exact and the behaviour we want.

---
 meta/classes/kernel-yocto.bbclass |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +1 -1
index b336e43..962b493 100644
@@ -20,7 +20,7 @@ def find_sccs(d):
     sources_list=[]
     for s in sources:
         base, ext = os.path.splitext(os.path.basename(s))
-        if ext and ext in ('.scc' '.cfg'):
+        if ext and ext in [".scc", ".cfg"]:
             sources_list.append(s)
         elif base and base in 'defconfig':
             sources_list.append(s)
Signature · 1 line
1.7.10.4
#4 lore raw

PATCH 4/5 kernel-yocto: allow multiple / shared kernel feature directories

In reply to Bruce Ashfield

To promote the reuse and sharing of configuration fragments this change allows any kernel-yocto based recipe to have multiple alternate git repositories which provide kernel feature directory trees listed on the SRC_URI.

These feature directories are in addition to any in-tree kernel meta data branches that may be available (described via the KMETA variable in linux-yocto recipes).

Features found within these directories can be used from recipes via the KERNEL_FEATURES variable. Features found within a feature directory are free to include any other features that are available in any directories. In both cases the path to a feature description (a .scc file) is relative to the root of a given feature directory (which is how existing .scc files work)

The search order for features is determined by the order that repositories appear on the SRC_URI.

Normal SRC_URI rules apply to any repository that is added as a kernel feature container. A SRCREV must be supplied and it must be unpacked to a unique directory, which is controlled via the "destsuffic" url parameter.

In addition to these standard requirements, any kernel feature repository reference should identify itself via the "type=kmeta" url parameter. If type=kmeta is not supplied, the repository will not be processed for kernel features.

As an example, the following in a linux-yocto bbappend makes two additional feature directories available to KERNEL_FEATURES and fragments.

SRC_URI += "git://git.yoctoproject.org/yocto-kernel-cache;protocol=git;branch=master;type=kmeta;name=feat1;destsuffix=kernel-cache/" SRC_URI += "git://${KSRC_linux_yocto_3_4};protocol=file;branch=meta;name=feat2;type=kmeta;destsuffix=kernel-features-experimental/"

SRCREV_feat1 = "${AUTOREV}"
SRCREV_feat2 = "${AUTOREV}"

---
 meta/classes/kernel-yocto.bbclass                  |   33 +++++++++++++++++++-
 .../kern-tools/kern-tools-native_git.bb            |    2 +-
 2 files changed, 33 insertions(+), 2 deletions(-)
meta/classes/kernel-yocto.bbclass +32 -1
index 962b493..92ede6a 100644
@@ -42,6 +42,23 @@ def find_urls(d):
 
     return patch_list
 
+# check the SRC_URI for "kmeta" type'd git repositories. Return the name of
+# the repository as it will be found in WORKDIR
+def find_kernel_feature_dirs(d):
+    feature_dirs=[]
+    fetch = bb.fetch2.Fetch([], d)
+    for url in fetch.urls:
+        urldata = fetch.ud[url]
+        parm = urldata.parm
+        if "type" in parm:
+            type = parm["type"]
+        if "destsuffix" in parm:
+            destdir = parm["destsuffix"]
+            if type == "kmeta":
+                feature_dirs.append(destdir)
+	    
+    return feature_dirs
+
 
 do_patch() {
 	cd ${S}
@@ -72,6 +89,7 @@ do_patch() {
 
 	sccs="${@" ".join(find_sccs(d))}"
 	patches="${@" ".join(find_patches(d))}"
+	feat_dirs="${@" ".join(find_kernel_feature_dirs(d))}"
 
 	set +e
 	# add any explicitly referenced features onto the end of the feature
@@ -82,13 +100,26 @@ do_patch() {
 		done
 	fi
 
+	# check for feature directories/repos/branches that were part of the
+	# SRC_URI. If they were supplied, we convert them into include directives
+	# for the update part of the process
+	if [ -n "${feat_dirs}" ]; then
+	    for f in ${feat_dirs}; do
+		if [ -d "${WORKDIR}/$f/meta" ]; then
+		    includes="$includes -I${WORKDIR}/$f/meta"
+		elif [ -d "${WORKDIR}/$f" ]; then
+		    includes="$includes -I${WORKDIR}/$f"
+		fi
+	    done
+	fi
+
 	if [ "${kbranch}" != "${KBRANCH_DEFAULT}" ]; then
 		updateme_flags="--branch ${kbranch}"
 	fi
 
 	# updates or generates the target description
 	updateme ${updateme_flags} -DKDESC=${KMACHINE}:${LINUX_KERNEL_TYPE} \
-                           ${addon_features} ${ARCH} ${KMACHINE} ${sccs} ${patches}
+                         ${includes} ${addon_features} ${ARCH} ${KMACHINE} ${sccs} ${patches}
 	if [ $? -ne 0 ]; then
 		echo "ERROR. Could not update ${kbranch}"
 		exit 1
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1
index 635dfe6..98cd8b9 100644
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
 DEPENDS = "git-native guilt-native"
 
-SRCREV = "62c8807825ea923a18b1519375b0df67ad9daf2b"
+SRCREV = "9284af9b968d40e441b10f5c09961cbe329ccb9b"
 PR = "r12"
 PV = "0.1+git${SRCPV}"
 
Signature · 1 line
1.7.10.4
#5 lore raw

PATCH 5/5 kernel-yocto/linux-yocto-custom: support low bandwidth options

In reply to Bruce Ashfield

To support configurations where active development is not being done within the oe/bitbake build environment and restricted bandwidth situations, this commit allows the SRC_URI to point to a kernel tgz instead of a full git repository.

Outside of the upstream tgz instead of a kernel git repository, the restrictions, config and patch process is the same as any linux-yocto-custom recipe.

An example linux-yocto-custom based recipe would have a configuration like this to build the 3.7 kernel, using an externally supplied config, from the 3.7 tgz:

  SRC_URI = "http://kernel.org/pub/linux/kernel/v3.0/linux-3.7.tar.bz2"
  PV = "3.7"
  S = "${WORKDIR}/linux-3.7"
  SRC_URI[md5sum] = "5323f3faadd051e83af605a63be5ea2e"
  SRC_URI[sha256sum] = "dc08d87a579fe2918362e6666e503a95a76296419195cb499aa9dd4dbe171a9e"

[YOCTO #2686]

---
 meta/classes/kernel-yocto.bbclass |   34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)
meta/classes/kernel-yocto.bbclass +21 -13
index 92ede6a..8494c16 100644
@@ -158,22 +158,10 @@ do_kernel_checkout() {
 		mkdir -p ${S}
 		
 		# We can fix up the kernel repository even if it wasn't a bare clone.
-		# If KMETA is defined, the branch must exist, but a machine branch
-		# can be missing since it may be created later by the tools.
 		mv ${WORKDIR}/git/.git ${S}
 		rm -rf ${WORKDIR}/git/
 		cd ${S}
-		if [ -n "${KMETA}" ]; then
-			git branch -a | grep -q ${KMETA}
-			if [ $? -ne 0 ]; then
-				echo "ERROR. The branch '${KMETA}' is required and was not"
-				echo "found. Ensure that the SRC_URI points to a valid linux-yocto"
-				echo "kernel repository"
-				exit 1
-			fi
-		fi
-	fi
-	if [ -d "${WORKDIR}/git/" ] && [ ! -d "${WORKDIR}/git/.git" ]; then
+	elif [ -d "${WORKDIR}/git/" ] && [ ! -d "${WORKDIR}/git/.git" ]; then
 		# we build out of {S}, so ensure that ${S} is clean and present
 		rm -rf ${S}
 		mkdir -p ${S}/.git
@@ -182,9 +170,29 @@ do_kernel_checkout() {
 		rm -rf ${WORKDIR}/git/
 		cd ${S}	
 		git config core.bare false
+	else
+	        # We have no git repository at all. To support low bandwidth options
+		# for building the kernel, we'll just convert the tree to a git repo
+	        # and let the rest of the process work unchanged
+	        cd ${S}
+		git init
+		git add .
+		git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
 	fi
 	# end debare
 
+       	# If KMETA is defined, the branch must exist, but a machine branch
+	# can be missing since it may be created later by the tools.
+	if [ -n "${KMETA}" ]; then
+		git branch -a | grep -q ${KMETA}
+		if [ $? -ne 0 ]; then
+			echo "ERROR. The branch '${KMETA}' is required and was not"
+			echo "found. Ensure that the SRC_URI points to a valid linux-yocto"
+			echo "kernel repository"
+			exit 1
+		fi
+	fi
+
 	# convert any remote branches to local tracking ones
 	for i in `git branch -a | grep remotes | grep -v HEAD`; do
 		b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
Signature · 1 line
1.7.10.4