Skip to contentSkip to conversation

classes/kernel-yocto: Apply patches from BSP

PATCH In review Joshua Watt 2 participants 11 messages Dec 19, 2019 – Dec 20, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 classes/kernel-yocto: Apply patches from BSP

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

---
 meta/classes/kernel-yocto.bbclass | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -5
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
 		if [ $? -ne 0 ]; then
 			bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
 		fi
-	fi
 
-	# run2: only generate patches for elements that have been passed on the SRC_URI
-	elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-	if [ -n "${elements}" ]; then
-		scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+		# run2: generate patches
+		scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
 		if [ $? -ne 0 ]; then
 			bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
 		fi
Signature · 1 line
2.23.0
#1 lore raw
Quoted 7 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Bruce

Quoted 33 lines
---
 meta/classes/kernel-yocto.bbclass | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                if [ $? -ne 0 ]; then
                        bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                if [ $? -ne 0 ]; then
                        bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                fi
--

2.23.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#2 lore raw
Quoted 10 lines · Bruce Ashfield
Quoted 7 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 6 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so,

s/userspace/recipespace/

Bruce

Quoted 44 lines

there are examples of this and I can dig them up and send them along.

Bruce

Quoted 28 lines
---
 meta/classes/kernel-yocto.bbclass | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                if [ $? -ne 0 ]; then
                        bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                if [ $? -ne 0 ]; then
                        bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                fi
--

2.23.0

Quoted 5 lines

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
#3 lore raw
In reply to Bruce Ashfield
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 37 lines

Bruce

Quoted 32 lines
---
  meta/classes/kernel-yocto.bbclass | 7 ++-----
  1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
--

2.23.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#4 lore raw
Quoted 18 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 31 lines
Quoted 31 lines

Bruce

Quoted 27 lines
---
  meta/classes/kernel-yocto.bbclass | 7 ++-----
  1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
--

2.23.0

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#5 lore raw
Quoted 30 lines · Bruce Ashfield
Quoted 18 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 7 lines

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I think they are standalone (e.g. they don't include any of the ktype files).

Quoted 8 lines

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

Quoted 8 lines

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

Does that mean the BSP scc files need to be moved from our metadata repo to live alongside the recipe, or is there some special sauce that allows the scc to be specified in SRC_URI, but pulled from our kmeta repo?

Quoted 66 lines

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 31 lines
Quoted 31 lines

Bruce

Quoted 27 lines
---
  meta/classes/kernel-yocto.bbclass | 7 ++-----
  1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
--

2.23.0

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#6 lore raw
Quoted 60 lines · Joshua Watt
Quoted 30 lines · Bruce Ashfield
Quoted 18 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 7 lines

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I think they are standalone (e.g. they don't include any of the ktype files).

Quoted 8 lines

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

Quoted 8 lines

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

Quoted 4 lines

Does that mean the BSP scc files need to be moved from our metadata repo to live alongside the recipe, or is there some special sauce that allows the scc to be specified in SRC_URI, but pulled from our kmeta repo?

At the moment, yes. Since you need to specify it in SRC_URI where the fetcher can find it (obviously :D), which implies that it isn't in that separate kernel meta repository.

What, if anything, do you currently have on your SRC_URI ? Just the kernel repo and your kernel meta data repo ?

Let me see if I can come up with a simple way (probably patch) to avoid that.

Bruce

Quoted 63 lines
Quoted 63 lines

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 38 lines
Quoted 38 lines

Bruce

Quoted 32 lines
---
  meta/classes/kernel-yocto.bbclass | 7 ++-----
  1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                 if [ $? -ne 0 ]; then
                         bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                 fi
--

2.23.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#7 lore raw
Quoted 57 lines · Bruce Ashfield
Quoted 52 lines · Joshua Watt
Quoted 26 lines · Bruce Ashfield
Quoted 16 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 7 lines

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I think they are standalone (e.g. they don't include any of the ktype files).

Quoted 7 lines

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

Quoted 8 lines

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

Quoted 4 lines

Does that mean the BSP scc files need to be moved from our metadata repo to live alongside the recipe, or is there some special sauce that allows the scc to be specified in SRC_URI, but pulled from our kmeta repo?

Quoted 6 lines

At the moment, yes. Since you need to specify it in SRC_URI where the fetcher can find it (obviously :D), which implies that it isn't in that separate kernel meta repository.

What, if anything, do you currently have on your SRC_URI ? Just the kernel repo and your kernel meta data repo ?

Yes, that is all we have.

Quoted 2 lines

Let me see if I can come up with a simple way (probably patch) to avoid that.

That would be great! Thanks.

Quoted 70 lines

Bruce

Quoted 57 lines
Quoted 57 lines

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 35 lines
Quoted 35 lines

Bruce

Quoted 32 lines
---
   meta/classes/kernel-yocto.bbclass | 7 ++-----
   1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
--

2.23.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#8 lore raw
Quoted 67 lines · Joshua Watt
Quoted 57 lines · Bruce Ashfield
Quoted 52 lines · Joshua Watt
Quoted 26 lines · Bruce Ashfield
Quoted 16 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 7 lines

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I think they are standalone (e.g. they don't include any of the ktype files).

Quoted 7 lines

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

Quoted 8 lines

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

Quoted 4 lines

Does that mean the BSP scc files need to be moved from our metadata repo to live alongside the recipe, or is there some special sauce that allows the scc to be specified in SRC_URI, but pulled from our kmeta repo?

Quoted 6 lines

At the moment, yes. Since you need to specify it in SRC_URI where the fetcher can find it (obviously :D), which implies that it isn't in that separate kernel meta repository.

What, if anything, do you currently have on your SRC_URI ? Just the kernel repo and your kernel meta data repo ?

Quoted 7 lines

Yes, that is all we have.

Quoted 2 lines

Let me see if I can come up with a simple way (probably patch) to avoid that.

That would be great! Thanks.

Attached is my proposed solution. Hopefully the commit log is detailed enough to explain what I'm trying to do :D

I've only tested that it doesn't break a standard BSP build, so let me know if this solves your use case, and I'll run some additional tests and send it to the list.

Bruce

Quoted 70 lines
Quoted 70 lines

Bruce

Quoted 57 lines
Quoted 57 lines

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 35 lines
Quoted 35 lines

Bruce

Quoted 32 lines
---
   meta/classes/kernel-yocto.bbclass | 7 ++-----
   1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
--

2.23.0

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

From 4383dac26536cb4c26f399edfa7faa0825c84cef Mon Sep 17 00:00:00 2001
From: Bruce Ashfield <[email protected]>
Date: Thu, 19 Dec 2019 17:07:18 -0500
Subject: [PATCH] kernel-yocto: allow external (aka non-integrated) BSPs to
supply patches

By default, only explicitly specified .scc files or patches are applied to a BSP as part of the patching phase. This allows the reference BSPs to be integrated into a kernel tree, and use the same meta data for tree generation and for runtime building. It also greatly simplies the processing of kernel meta data.

That being said, if an external (and fully standalone) kmeta repository is used, it very well may contain both patches and configuration. To allow the BSP definition to supply patches and config, we introduce a new kernel meta data definition:

define KMETA_EXTERNAL_BSP t

This can be set in any .scc file that wants to specify both patches and configuration.

If specifying this per-BSP is not desired, a secondary variable is used:

KMETA_EXTERNAL_BSPS

When that is set in a bbappend, any located BSP definitions will be used for both patches and configuration.

---
 meta/classes/kernel-yocto.bbclass | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
meta/classes/kernel-yocto.bbclass +15 -0
index 6537cdf416..e367fe3e26 100644
@@ -220,6 +220,14 @@ do_kernel_metadata() {
 		if [ $? -ne 0 ]; then
 			bbfatal_log "Could not locate BSP definition for ${KMACHINE}/${LINUX_KERNEL_TYPE} and no defconfig was provided"
 		fi
+
+		# if the bsp definition has "define KMETA_EXTERNAL_BSP t",
+		# then we need to set a flag that will instruct the next
+		# steps to use the BSP as both configuration and patches.
+		grep -q KMETA_EXTERNAL_BSP $bsp_definition
+		if [ $? -eq 0 ]; then
+		    KMETA_EXTERNAL_BSPS="t"
+		fi
 	fi
 	meta_dir=$(kgit --meta)
 
@@ -233,6 +241,13 @@ do_kernel_metadata() {
 		fi
 	fi
 
+	# if KMETA_EXTERNAL_BSPS has been set, or it has been detected from
+	# the bsp definition, then we inject the bsp_definition into the
+	# patch phase below.  we'll piggy back on the sccs variable.
+	if [ -n "${KMETA_EXTERNAL_BSPS}" ]; then
+		sccs = "${bsp_definition} ${sccs}"
+	fi
+
 	# run2: only generate patches for elements that have been passed on the SRC_URI
 	elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
 	if [ -n "${elements}" ]; then
Signature · 1 line
2.19.1
#9 lore raw
Quoted 77 lines · Bruce Ashfield
Quoted 67 lines · Joshua Watt
Quoted 57 lines · Bruce Ashfield
Quoted 52 lines · Joshua Watt
Quoted 26 lines · Bruce Ashfield
Quoted 16 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 7 lines

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I think they are standalone (e.g. they don't include any of the ktype files).

Quoted 7 lines

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

Quoted 8 lines

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

Quoted 4 lines

Does that mean the BSP scc files need to be moved from our metadata repo to live alongside the recipe, or is there some special sauce that allows the scc to be specified in SRC_URI, but pulled from our kmeta repo?

Quoted 6 lines

At the moment, yes. Since you need to specify it in SRC_URI where the fetcher can find it (obviously :D), which implies that it isn't in that separate kernel meta repository.

What, if anything, do you currently have on your SRC_URI ? Just the kernel repo and your kernel meta data repo ?

Quoted 7 lines

Yes, that is all we have.

Quoted 2 lines

Let me see if I can come up with a simple way (probably patch) to avoid that.

That would be great! Thanks.

Quoted 6 lines

Attached is my proposed solution. Hopefully the commit log is detailed enough to explain what I'm trying to do :D

I've only tested that it doesn't break a standard BSP build, so let me know if this solves your use case, and I'll run some additional tests and send it to the list.

I had to make one minor fix to your patch (the assignment of the sccs variable needs the spaces around the "=" removed to be valid shell code), but other than that your patch works great for me and does exactly what I want.

Thanks!

Quoted 80 lines

Bruce

Quoted 66 lines
Quoted 66 lines

Bruce

Quoted 61 lines
Quoted 61 lines

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 29 lines
Quoted 29 lines

Bruce

Quoted 27 lines
---
   meta/classes/kernel-yocto.bbclass | 7 ++-----
   1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
--

2.23.0

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

#10 lore raw
Quoted 82 lines · Joshua Watt
Quoted 80 lines · Bruce Ashfield
Quoted 67 lines · Joshua Watt
Quoted 57 lines · Bruce Ashfield
Quoted 52 lines · Joshua Watt
Quoted 26 lines · Bruce Ashfield
Quoted 16 lines · Joshua Watt
Quoted 7 lines · Bruce Ashfield
Quoted 6 lines · Joshua Watt

0f698dfd1c8 ("kernel-yocto: streamline patch, configuration and audit phases") appears to have inadvertently broken support for BSP .scc files to provide patches for the kernel. Restore this behavior by adding $bsp_definition to the list of files processed for patches. Additionally, the logic can be simplified now that the same elements are used for both configuration fragments and patches

Quoted 7 lines

Unfortunately no, we can't do this.

It would reapply all the patches that are already integrated into the tree, and that distinction is the key part of that change.

What exactly are you trying to do ? Define a userspace BSP ? If so, there are examples of this and I can dig them up and send them along.

Quoted 9 lines

We have a lot of boards running different SoCs (each with a different vendor provided kernel) and we are trying to keep our sanity managing all of the by using a metadata repo. So for example, we have a recipe that pulls in vendor A's kernel directly from the upstream source, then our local metadata repo which provides the scc files we need to configure that repo. Inevitably, we end up needing to patch these kernels, so I was hoping that we could provide the patches in the metadata repo and add them to the BSP .scc file, but that wasn't working.

Quoted 7 lines

Right! It does make sense to keep the configs and patches together in that meta data repo.

So you have your own kmeta repo, and that's where the BSP definitions that are being searched and found. Are those definitions completely standalone, or are they including some of the common yocto ktype definitions ? (It makes a difference to how they can be included).

I think they are standalone (e.g. they don't include any of the ktype files).

Quoted 7 lines

I explained the details of the behaviour in an email on Oct 9th, with the title: "[yocto] Kernel patches pulled from BSP definition file are not applied.", so I won't repeat that long description here .. except to say that it is working as intended (but that doesn't mean it is set in stone .. see below ..). And yes, there are doc updates that need to happen for this to more clearly explain the design and options.

Quoted 8 lines

That being said, there are a couple of things that can be done, and one that works right now:

- The BSP definition SCC needs to be on the SRC_URI itself (as I describe in that other email), if your kernel recipe's have different names, that's doable. If they are under a common recipe, that of course would be harder (but you could use a variable for the MACHINE and have it work that way).

Quoted 4 lines

Does that mean the BSP scc files need to be moved from our metadata repo to live alongside the recipe, or is there some special sauce that allows the scc to be specified in SRC_URI, but pulled from our kmeta repo?

Quoted 6 lines

At the moment, yes. Since you need to specify it in SRC_URI where the fetcher can find it (obviously :D), which implies that it isn't in that separate kernel meta repository.

What, if anything, do you currently have on your SRC_URI ? Just the kernel repo and your kernel meta data repo ?

Quoted 7 lines

Yes, that is all we have.

Quoted 2 lines

Let me see if I can come up with a simple way (probably patch) to avoid that.

That would be great! Thanks.

Attached is my proposed solution. Hopefully the commit log is detailed enough to explain what I'm trying to do :D

Quoted 9 lines
Quoted 3 lines

I've only tested that it doesn't break a standard BSP build, so let me know if this solves your use case, and I'll run some additional tests and send it to the list.

I had to make one minor fix to your patch (the assignment of the sccs variable needs the spaces around the "=" removed to be valid shell code), but other than that your patch works great for me and does exactly what I want.

ahah. Right you are. I'll make that tweak and send it to the list.

Bruce

Quoted 82 lines

Thanks!

Quoted 80 lines

Bruce

Quoted 66 lines
Quoted 66 lines

Bruce

Quoted 61 lines
Quoted 61 lines

That's the one that works now.

A second option would be if I inject some logic to differentiate an integrated BSP versus one that requires patching. That's a slippery slope back into the complex logic that used to be in place, but I can think of a few viable ways to do it.

Definitely a use case we can support, it'll just take some tweaking.

Cheers,

Bruce

Bruce

Quoted 29 lines
Quoted 29 lines

Bruce

Quoted 27 lines
---
   meta/classes/kernel-yocto.bbclass | 7 ++-----
   1 file changed, 2 insertions(+), 5 deletions(-)
meta/classes/kernel-yocto.bbclass +2 -6
index ed9bcfa57c..08c02a8f50 100644
@@ -193,12 +193,9 @@ do_kernel_metadata() {
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
-       fi
 
-       # run2: only generate patches for elements that have been passed on the SRC_URI
-       elements="`echo -n ${sccs} ${patches} ${KERNEL_FEATURES}`"
-       if [ -n "${elements}" ]; then
-               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${sccs} ${patches} ${KERNEL_FEATURES}
+               # run2: generate patches
+               scc --force -o ${S}/${meta_dir}:patch --cmds patch ${includes} ${bsp_definition} ${sccs} ${patches} ${KERNEL_FEATURES}
                  if [ $? -ne 0 ]; then
                          bbfatal_log "Could not generate configuration queue for ${KMACHINE}."
                  fi
--

2.23.0

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Quoted 4 lines
Quoted 4 lines

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II