Skip to contentSkip to conversation

Rework merge of feature branches

PATCH In review David Vincent 3 participants 11 messages Jan 25, 2017 – Feb 7, 2017

lore
Share & tools
lore
#0 lore raw

cover Rework merge of feature branches

I've recently noticed that, on Morty, feature branches of a Yocto kernel are not merged anymore. This is due to the fact that the meta-series file does not process the git merge commands anymore.

This commit creates a new merge command to be executed when processing the kernel metadata. To solve the issue, there is also a patch to send to oe-core for kernel-yocto.bbclass, but I wait for this patch to be included since there is no point without it.

David Vincent (1):
scc: Move merge command

 tools/kgit-meta          | 14 ++++++++++++++
 tools/scc                | 13 -------------
 tools/scc-cmds/merge.cmd | 10 ++++++++++
 3 files changed, 24 insertions(+), 13 deletions(-)
 create mode 100644 tools/scc-cmds/merge.cmd
Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/1 scc: Move merge command

Create a new command to merge a feature branch inside the current branch when processing meta-series.

---
 tools/kgit-meta          | 14 ++++++++++++++
 tools/scc                | 13 -------------
 tools/scc-cmds/merge.cmd | 10 ++++++++++
 3 files changed, 24 insertions(+), 13 deletions(-)
 create mode 100644 tools/scc-cmds/merge.cmd
tools/kgit-meta +14 -0
index 1c19ef5..aa5e210 100755
@@ -254,6 +254,20 @@ for fline in `cat ${meta_series}`; do
                 fi
             fi
             ;;
+        merge:*)
+            b=$(echo ${fline} | cut -d: -f2 | sed 's%^\ %%')
+            check_branch ${b}
+            if [ $? -ne 0 ]; then
+                echo "ERROR. branch ${b} does not exist, can't merge"
+                exit 1
+            else
+                echo "[INFO]: branch merge: ${b}"
+                eval git merge -q --no-ff -m \"Merge branch ${b}\" ${b}
+                if [ $? -ne 0 ]; then
+                exit 1
+                fi
+            fi
+            ;;
         *)
             ;;
     esac
tools/scc +0 -13
index 3d17496..cee9e0d 100755
@@ -286,19 +286,6 @@ set_kernel_version()
     KERNEL_VERSION=$1
 }
 
-# Used in feature scripts
-#  - convenience wrapper for "git merge" 
-#  - arguments are git merge <args> and are passed directly to git
-merge()
-{
-    args="$@"
-
-    # call the git wrapper function. Currently with no modification or
-    # checking on the args, but the option of checking in the future is
-    # trivial once users have converted.
-    git merge $args
-}
-
 # used in feature scripts
 #  - encodes a generic git command
 git()
tools/scc-cmds/merge.cmd new file +10 -0
new file mode 100644
index 0000000..02be6ca
@@ -0,0 +1,10 @@
+# used in feature scripts
+#    arg1: the branch name
+merge()
+{
+    local text
+    local mbranch_name=$1
+
+    text="merge: ${mbranch_name}"
+    eval echo "\$text" $outfile_append
+}
Signature · 1 line
2.11.0
#2 lore raw
In reply to David Vincent
Quoted 5 lines · David Vincent

I've recently noticed that, on Morty, feature branches of a Yocto kernel are not
merged anymore. This is due to the fact that the meta-series file does not process the git merge commands anymore.

Indeed. I dropped that functionality on purpose when I did the simplification
and streamlining of the processing .. since I wanted to see if anyone was using it.

I now have my answer :D

I need to have a closer look at the patch and run it through some tests here, but on my glance, you've done it correctly.

Quoted 7 lines

This commit creates a new merge command to be executed when processing the kernel metadata. To solve the issue, there is also a patch to send to oe-core
for kernel-yocto.bbclass, but I wait for this patch to be included since there
is no point without it.

Send that patch to me, or I'll just do the patch myself .. since yes, we do need
to add merge to the list of valid comments when processing the meta data.

I'll submit all the changes in my next consolidated pull request for the kernels + tools.

Bruce

Quoted 19 lines

David Vincent (1):
scc: Move merge command

 tools/kgit-meta          | 14 ++++++++++++++
 tools/scc                | 13 -------------
 tools/scc-cmds/merge.cmd | 10 ++++++++++
 3 files changed, 24 insertions(+), 13 deletions(-)
 create mode 100644 tools/scc-cmds/merge.cmd

--
2.11.0

--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"
#3 lore raw

PATCH 1/1 kernel-yocto: Update source tree with kgit-meta

kgit-meta processes a file called meta-series to update the source tree (create, merge branches, ...). This fixes the merging of feature branches using the new merge command of yocto-kernel-tools.

---
 meta/classes/kernel-yocto.bbclass | 13 +++++++++++++
 1 file changed, 13 insertions(+)
meta/classes/kernel-yocto.bbclass +13 -0
index 5cfd8aff50..b37ac01a84 100644
@@ -162,6 +162,19 @@ do_kernel_metadata() {
 			bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
 		fi
 	fi
+
+	# Update git repository by running kgit-meta
+	cd ${S}
+
+	check_git_config
+	meta_dir=$(kgit --meta)
+	if [ -f "${meta_dir}/meta-series" ]; then
+		kgit-meta ${meta_dir}/meta-series
+		if [ $? -ne 0 ]; then
+			bberror "Could not apply metadata for ${KMACHINE}."
+			bbfatal_log "Failures can be resolved in the linux source directory ${S})"
+		fi
+	fi
 }
 
 do_patch() {
Signature · 1 line
2.11.0
#4 lore raw

Re: [PATCH] kernel-yocto: Update source tree with kgit-meta

Quoted 3 lines · David Vincent

kgit-meta processes a file called meta-series to update the source tree (create, merge branches, ...). This fixes the merging of feature branches using the new merge command of yocto-kernel-tools.

I have this queued, with some modifications. I as traveling this week, so it'll be early next week before this goes out for merge.

cheers,

Bruce

Quoted 31 lines
---
 meta/classes/kernel-yocto.bbclass | 13 +++++++++++++
 1 file changed, 13 insertions(+)
meta/classes/kernel-yocto.bbclass +13 -0
index 5cfd8aff50..b37ac01a84 100644
@@ -162,6 +162,19 @@ do_kernel_metadata() {
 			bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
 		fi
 	fi
+
+	# Update git repository by running kgit-meta
+	cd ${S}
+
+	check_git_config
+	meta_dir=$(kgit --meta)
+	if [ -f "${meta_dir}/meta-series" ]; then
+		kgit-meta ${meta_dir}/meta-series
+		if [ $? -ne 0 ]; then
+			bberror "Could not apply metadata for ${KMACHINE}."
+			bbfatal_log "Failures can be resolved in the linux source directory ${S})"
+		fi
+	fi
 }
 
 do_patch() {
#5 lore raw
In reply to Bruce Ashfield
Quoted 30 lines · Bruce Ashfield
Quoted 5 lines · David Vincent

I've recently noticed that, on Morty, feature branches of a Yocto kernel are not
merged anymore. This is due to the fact that the meta-series file does not process the git merge commands anymore.

Indeed. I dropped that functionality on purpose when I did the simplification
and streamlining of the processing .. since I wanted to see if anyone was using it.

I now have my answer :D

I need to have a closer look at the patch and run it through some tests here, but on my glance, you've done it correctly.

Quoted 7 lines

This commit creates a new merge command to be executed when processing the kernel metadata. To solve the issue, there is also a patch to send to oe-core
for kernel-yocto.bbclass, but I wait for this patch to be included since there
is no point without it.

Quoted 7 lines

Send that patch to me, or I'll just do the patch myself .. since yes, we do need
to add merge to the list of valid comments when processing the meta data.

I'll submit all the changes in my next consolidated pull request for the kernels + tools.

I spent Friday looking into this in more detail, and to keep things consistent
with how the path and config queues are handled, with all the tree manipulations
pull out of the various scripts and exposed in the bbclass itself, I ended up with
a different implementation of the support.

The kern-tools change is the same as you did (and is your change), the difference
is in the bbclass processing, where do_patch() modifies the tree, and it consumes
the output of the meta-data routine .. versus the kgit-meta script itself.

The work can be seen in my poky-contrib tree:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel,
in the
commit "kern-tools: re-enable scc merge command".

These changes are on master, and passed my testing to merge feature branches into the branch being built.

If you are able to try it out (on master), that would be great. I'm going to let it
soak for a few days, and once it merges to master, it can be ported to morty.

Cheers,

Bruce

Quoted 30 lines

Bruce

Quoted 10 lines

David Vincent (1):
scc: Move merge command

 tools/kgit-meta          | 14 ++++++++++++++
 tools/scc                | 13 -------------
 tools/scc-cmds/merge.cmd | 10 ++++++++++
 3 files changed, 24 insertions(+), 13 deletions(-)
 create mode 100644 tools/scc-cmds/merge.cmd
Quoted 9 lines

--
2.11.0

--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

--
"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"

Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end"
#6 lore raw
Quoted 11 lines · On dimanche 29 janvier 2017 23:31:49 CET Bruce Ashfield wrote:

On Wed, Jan 25, 2017 at 1:05 PM, Bruce Ashfield <[email protected]> wrote:

I spent Friday looking into this in more detail, and to keep things consistent
with how the path and config queues are handled, with all the tree manipulations
pull out of the various scripts and exposed in the bbclass itself, I ended up with
a different implementation of the support.

Quoted 5 lines

The kern-tools change is the same as you did (and is your change), the difference
is in the bbclass processing, where do_patch() modifies the tree, and it consumes
the output of the meta-data routine .. versus the kgit-meta script itself.

Thanks for looking into this and come up with a better implementation. I agree that mine was a little harsh.

Quoted 8 lines

The work can be seen in my poky-contrib tree:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel,
in the
commit "kern-tools: re-enable scc merge command".

These changes are on master, and passed my testing to merge feature branches into the branch being built.

Saw it, read it and saw that it was now in master-next, ready to be pulled.

Quoted 5 lines

If you are able to try it out (on master), that would be great. I'm going to let it
soak for a few days, and once it merges to master, it can be ported to morty.

Sorry, I don't have a master build ready but as soon as I can test it whether on morty or master, I will report back.

David

#7 lore raw
Quoted 27 lines · David Vincent
Quoted 11 lines · On dimanche 29 janvier 2017 23:31:49 CET Bruce Ashfield wrote:

On Wed, Jan 25, 2017 at 1:05 PM, Bruce Ashfield <[email protected]> wrote:

I spent Friday looking into this in more detail, and to keep things consistent
with how the path and config queues are handled, with all the tree manipulations
pull out of the various scripts and exposed in the bbclass itself, I ended up with
a different implementation of the support.

Quoted 5 lines

The kern-tools change is the same as you did (and is your change), the difference
is in the bbclass processing, where do_patch() modifies the tree, and it consumes
the output of the meta-data routine .. versus the kgit-meta script itself.

Thanks for looking into this and come up with a better implementation. I agree that mine was a little harsh.

Quoted 8 lines

The work can be seen in my poky-contrib tree:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel,
in the
commit "kern-tools: re-enable scc merge command".

These changes are on master, and passed my testing to merge feature branches into the branch being built.

Quoted 8 lines

Saw it, read it and saw that it was now in master-next, ready to be pulled.

Quoted 5 lines

If you are able to try it out (on master), that would be great. I'm going to let it
soak for a few days, and once it merges to master, it can be ported to morty.

Sorry, I don't have a master build ready but as soon as I can test it whether on morty or master, I will report back.

Perfect. Since I just sent my kernel series to master, I can flip my builder to morty and have a look as well. I'd still like to confirm that it does work in your scenario, but I can do the lifting to get it ready for morty with a run of my own.

Cheers,

Bruce

Quoted 3 lines

David

#8 lore raw
In reply to David Vincent
Quoted 27 lines · David Vincent
Quoted 11 lines · On dimanche 29 janvier 2017 23:31:49 CET Bruce Ashfield wrote:

On Wed, Jan 25, 2017 at 1:05 PM, Bruce Ashfield <[email protected]> wrote:

I spent Friday looking into this in more detail, and to keep things consistent
with how the path and config queues are handled, with all the tree manipulations
pull out of the various scripts and exposed in the bbclass itself, I ended up with
a different implementation of the support.

Quoted 5 lines

The kern-tools change is the same as you did (and is your change), the difference
is in the bbclass processing, where do_patch() modifies the tree, and it consumes
the output of the meta-data routine .. versus the kgit-meta script itself.

Thanks for looking into this and come up with a better implementation. I agree that mine was a little harsh.

Quoted 8 lines

The work can be seen in my poky-contrib tree:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=zedd/kernel,
in the
commit "kern-tools: re-enable scc merge command".

These changes are on master, and passed my testing to merge feature branches into the branch being built.

Quoted 8 lines

Saw it, read it and saw that it was now in master-next, ready to be pulled.

Quoted 5 lines

If you are able to try it out (on master), that would be great. I'm going to let it
soak for a few days, and once it merges to master, it can be ported to morty.

Sorry, I don't have a master build ready but as soon as I can test it whether on morty or master, I will report back.

I was able to merge branches with this patch applied to Morty.

Cheers,

Bruce

Quoted 3 lines

David

From 8e08efa4b567e58386247575ae28835841f484c0 Mon Sep 17 00:00:00 2001
From: Bruce Ashfield <[email protected]>
Date: Wed, 25 Jan 2017 13:29:34 -0500
Subject: [PATCH] kern-tools: re-enable scc merge command

The ability to merge two branches directly from a .scc file was dropped during the streamlining of the tools.

As was pointed out by David Vincent <[email protected]>, there is once again a valid use case for this functionality, so we restore the capability.

---
 meta/classes/kernel-yocto.bbclass                       | 16 +++++++++++++++-
 meta/recipes-kernel/kern-tools/kern-tools-native_git.bb |  3 ++-
 2 files changed, 17 insertions(+), 2 deletions(-)
meta/classes/kernel-yocto.bbclass +15 -1
index a60327a..eb35dce 100644
@@ -148,7 +148,7 @@ do_kernel_metadata() {
 	# run1: pull all the configuration fragments, no matter where they come from
 	elements="`echo -n ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}`"
 	if [ -n "${elements}" ]; then
-		scc --force -o ${S}/${meta_dir}:cfg,meta ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
+		scc --force -o ${S}/${meta_dir}:cfg,merge,meta ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
 		if [ $? -ne 0 ]; then
 			bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
 		fi
@@ -165,6 +165,7 @@ do_kernel_metadata() {
 }
 
 do_patch() {
+	set +e
 	cd ${S}
 
 	check_git_config
@@ -177,6 +178,19 @@ do_patch() {
 			bbfatal_log "Patch failures can be resolved in the linux source directory ${S})"
 		fi
 	fi
+
+	if [ -f "${meta_dir}/merge.queue" ]; then
+		# we need to merge all these branches
+		for b in $(cat ${meta_dir}/merge.queue); do
+			git show-ref --verify --quiet refs/heads/${b}
+			if [ $? -eq 0 ]; then
+				bbnote "Merging branch ${b}"
+				git merge -q --no-ff -m "Merge branch ${b}" ${b}
+			else
+				bbfatal "branch ${b} does not exist, cannot merge"
+			fi
+		done
+	fi
 }
 
 do_kernel_checkout() {
meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +2 -1
index 8af0155..aa27ec8 100644
@@ -4,7 +4,8 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=a6c2fa8aef1b
 
 DEPENDS = "git-native"
 
-SRCREV = "85564e69555b713c2759d58ec5ade54424d051d8"
+SRCREV = "c14440d4e7ae0160c260ed65c3e123be5dc97ae8"
+
 PR = "r12"
 PV = "0.2+git${SRCPV}"
 
Signature · 1 line
2.7.4
#9 lore raw
Quoted 7 lines · On vendredi 3 février 2017 13:51:34 CET Bruce Ashfield wrote:

I was able to merge branches with this patch applied to Morty.

Cheers,

Bruce

I've just successfully tested it on Morty myself. Works for me too, branches are merged just like they were before !

Thank you
David

#10 lore raw
Quoted 10 lines · David Vincent
Quoted 7 lines · On vendredi 3 février 2017 13:51:34 CET Bruce Ashfield wrote:

I was able to merge branches with this patch applied to Morty.

Cheers,

Bruce

I've just successfully tested it on Morty myself. Works for me too, branches are merged just like they were before !

Good news!

I'll send the patch to the morty maintainer so it can be merged.

Cheers,

Bruce

Quoted 5 lines

Thank you
David