linux-yocto: improve checkout error handling and reporting

Richard/Saul,

I've got a long description in the patch itself that gives most of the details of this change. But I'll hit some extra points here.

I've been cooking up this change for quite a while, since people seem to be working with local repositories and it is a common mistake to point at a non-bare linux-yocto repo. Due to the way the WORKDIR repo is created, this can cause some really cryptic build errors. This change addresses that issue and adds some workflow improvements at the same time.

- it can optionally work with the 'bareclone' option in a SRC_URI. I sent that patch to bitbake-devel today. This isn't required, but if it is used, a full bare clone is made in WORKDIR, and the checkout phase of linux-yocto detects that and creates local tracking branches for the remotes. Mixes of local and remote branches work perfectly here.

- it can also detect if a local, non-bare repository was the src repo. In this case, the tree is validated for the minimum branches that make the rest of the build viable, and if they aren't found a clear message is dumped about the missing branches. No cryptic errors from the build.

- instead of poking at git internals to create the local branches in WORKDIR. real tracking branches are created. With this, you now have the ability to work on your repository and push changes to your upstream, or fetch changes into the remotes and then bring them into your local branches. Everthing is setup properly for a more 'normal' development workflow.

I wanted to send this out before I send my -stable updates to the repos themselves.

Cheers,

Bruce

The following changes since commit d8b3718303c3d56394c722059f1a73bd79131d0a:

texi2html: Fix for multilib (2012-02-22 23:03:16 +0000)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib zedd/kernel
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (1):
linux-yocto: improve checkout error handling and reporting

 meta/classes/kernel-yocto.bbclass |   72 +++++++++++++++++++++++++------------
 1 files changed, 49 insertions(+), 23 deletions(-)

1.7.4.1

The typical workflow for linux-yocto simply uses a remote upstream repository (Whether it is mirrored or not), and in this case there are no issues with consistency in the format of the resository that is unpacked into the WORKDIR.

When working with a local linux-yocto repository for kernel development the remote vs local branches is not always consistent between repositories.

The suggested/documented workflow has always been to use a bare clone of linux-yocto, and use a second working tree repository for development. Changes flow from the working tree to the bare clone and then into the working directory for build. A common mistake that happens with this workflow is that the non-bare, working repository is used instead of the bare clone version.

If a non-bare repository is reference by the SRC_URI, then the branches that are fetched into WORKDIR are not consitent. If the MACHINE and META branches are not present, cryptic build errors will result.

To solve this problem, the checkout code has been changed in several ways:

- works with a newly proposed 'bareclone' option to bitbake
- detects if a bareclone is present in WORKDIR or not and adjustst the checkout accordingly.
- if a non-bare clone is detected, machine and meta branches are checked. If they are not present, or can't be created a clear error message is produced
- instead of manipulating the refs directly in the git tree, local tracking branches are (quietly) created for remote branches. Enabling a better workflow in the WORKDIR kernel repository.

This has been tested with linux-yocto remote upstreams, local bare and non-bare respositories. All builds succeed or fail with clear error messages.

---
 meta/classes/kernel-yocto.bbclass |   72 +++++++++++++++++++++++++------------
 1 files changed, 49 insertions(+), 23 deletions(-)

index 3130bf4..50ff555 100644
@@ -139,33 +139,59 @@ do_patch() {
 }
 
 do_kernel_checkout() {
-	if [ -d ${WORKDIR}/git/.git/refs/remotes/origin ]; then
-		echo "Fixing up git directory for ${LINUX_KERNEL_TYPE}/${KMACHINE}"
-		rm -rf ${S}
-		mkdir ${S}
+	# we build out of {S}, so ensure that ${S} is clean and present
+	rm -rf ${S}
+	mkdir -p ${S}/.git
+
+	set +e
+
+	# A linux yocto SRC_URI should use the bareclone option. That
+	# ensures that all the branches are available in the WORKDIR version
+	# of the repository. If it wasn't passed, we should detect it, and put
+	# out a useful error message
+	if [ -d "${WORKDIR}/git/.git" ]; then
+		echo "WARNING. ${WORKDIR}/git is not a bare clone."
+		echo "Ensure that the SRC_URI includes the 'bareclone=1' option."
+		
+		# we can fix up the kernel repository, but at the least the meta
+		# branch must be present. The machine branch may be created later.
 		mv ${WORKDIR}/git/.git ${S}
-	
-		if [ -e ${S}/.git/packed-refs ]; then
-			cd ${S}
-			rm -f .git/refs/remotes/origin/HEAD
-IFS='
-';
-			for r in `git show-ref | grep remotes`; do
-				ref=`echo $r | cut -d' ' -f1`; 
-				b=`echo $r | cut -d' ' -f2 | sed 's%refs/remotes/origin/%%'`;
-				dir=`dirname $b`
-				mkdir -p .git/refs/heads/$dir
-				echo $ref > .git/refs/heads/$b
-			done
-			cd ..
-		else
-			cp -r ${S}/.git/refs/remotes/origin/* ${S}/.git/refs/heads
-			rmdir ${S}/.git/refs/remotes/origin
+		rm -rf ${WORKDIR}/git/
+		cd ${S}
+		git branch -a | grep -q ${KMETA}
+		if [ $? -ne 0 ]; then
+			echo "ERROR. The branch '${KMETA}' is required and was not"
+			echo "found. Ensure that the SRC_URI points to a valid linux-yocto"
+			echo "kernel repository"
+			exit 1
 		fi
+	 	if [ -z "${YOCTO_KERNEL_EXTERNAL_BRANCH}" ]; then
+			git branch -a | grep -q ${KBRANCH}
+			if [ $? -ne 0 ]; then
+				echo "ERROR. The branch '${KBRANCH}' is required and was not"
+				echo "found. Ensure that the SRC_URI points to a valid linux-yocto"
+				echo "kernel repository"
+				exit 1
+			fi
+		fi
+	else
+		mv ${WORKDIR}/git/* ${S}/.git
+		rm -rf ${WORKDIR}/git/
+		cd ${S}
+		git config core.bare false
 	fi
-	cd ${S}
+	# end debare
 
-	set +e
+	# convert any remote branches to local tracking ones
+	for i in `git branch -a | grep remotes | grep -v HEAD`; do
+		b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
+		git show-ref --quiet --verify -- "refs/heads/$b"
+		if [ $? -ne 0 ]; then
+			git branch $b $i > /dev/null
+		fi
+	done
+
+	# Create a working tree copy of the kernel by checkout out a branch
 	git show-ref --quiet --verify -- "refs/heads/${KBRANCH}"
 	if [ $? -eq 0 ]; then
 		# checkout and clobber and unimportant files

1.7.4.1