linux-yocto + kern tools consolidated pull request

Updating the SRCREV to import the following changes.

[updateme: find the board description with the highest score]

This removes the requirement that a custom linux-yocto .scc file have define KTYPE <foo>, where <foo> is typically "standard". The tools can now match on a .scc file that only matches the board, but will still chose one that matches the board and kernel type, if available.

[updateme: allow for tabs or spaces in defines]

define KMACHINE<tab>$MACHINE was missed by the regex.

[scc/kgit-meta: detect and avoid duplicating patching]

To allow feature description to be included multiple times, they were previously split into -enable and 'patch' descriptions. With this change the patches will be detected as already included, and skipped automatically. Removing the need to do this split. It also cleans up the ability to warn about multiple includes.

[kconf_check: add "verify" configuration fragment type]

This adds the ability for a BSP to have a kernel configuration fragment that lists options that must be present. If they are not present it is a hard error. "required" is a similar fragment, but it adds them to the build, and audits them at the end, but does not abort the build if they are present. This is a minor distinction, but one that is useful when creating flexible, shared kernel config structures.

[kconf_check: improve kernel audit report formatting] [kconf_check: perform validity checks on non-hardware options] [kconf_check: cleanups and verbose flag]

The existing output was verbose and not always useful to the reader. This change makes the output more compact, audits non-hardware options and gives information

[invalid (54)]: meta/cfg/preempt-rt/common-pc/invalid.cfg
This BSP sets config options that are not offered anywhere within this kernel

Signed-off-by Bruce Ashfield <[email protected]>

---
 meta/recipes-kernel/kern-tools/kern-tools-native_git.bb |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1

index ce94885..f2cd39f 100644
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
 DEPENDS = "git-native guilt-native"
 
-SRCREV = "a802ee9c8d9334c0f7932dfd40d45599addb7c90"
+SRCREV = "6f68c9473b43c3e39755a72aef8733cbd0bf1a59"
 PR = "r12"
 PV = "0.1+git${SRCPV}"
 

Quoted 4 lines · Bruce Ashfield

Updating the SRCREV to import the following changes.

[updateme: find the board description with the highest score]

Quoted 4 lines

This removes the requirement that a custom linux-yocto .scc file have define KTYPE <foo>, where <foo> is typically "standard". The tools can now match on a .scc file that only matches the board, but will still chose one that matches the board and kernel type, if available.

Should the documentation then state that KTYPE is only necessary to define if it is not "standard" ? Does this also apply to

linux-yocto-custom recipes/kernels where the repository could very well not define any ktype at all?

Quoted 16 lines

[updateme: allow for tabs or spaces in defines]

define KMACHINE<tab>$MACHINE was missed by the regex.

[scc/kgit-meta: detect and avoid duplicating patching]

To allow feature description to be included multiple times, they were previously split into -enable and 'patch' descriptions. With this change the patches will be detected as already included, and skipped automatically. Removing the need to do this split. It also cleans up the ability to warn about multiple includes.

[kconf_check: add "verify" configuration fragment type]

Quoted 7 lines

This adds the ability for a BSP to have a kernel configuration fragment that lists options that must be present. If they are not present it is a hard error. "required" is a similar fragment, but it adds them to the build, and audits them at the end, but does not abort the build if they are present. This is a minor distinction, but one that is useful when creating flexible, shared kernel config structures.

IIRC we discussed this verify thing at ELCE and how it broke some of the semantics.... trying to remember now, let's see:

kconf hardware foo.cfg
kconf verify hardware bar.cfg
kconf non-hardware foobar.cfg
kconf verify non-hardware barfoo.cfg

Is that how this is to be used? The configuration space just doubled from a documentation point of view, and that is without even considering the "required" keyword you described.

Can you use required with verify? Can you use both of them with both hardware and non-hardware?

Quoted 33 lines

[kconf_check: improve kernel audit report formatting] [kconf_check: perform validity checks on non-hardware options] [kconf_check: cleanups and verbose flag]

The existing output was verbose and not always useful to the reader. This change makes the output more compact, audits non-hardware options and gives information

[invalid (54)]: meta/cfg/preempt-rt/common-pc/invalid.cfg
This BSP sets config options that are not offered anywhere within this kernel

Signed-off-by Bruce Ashfield <[email protected]>

---
  meta/recipes-kernel/kern-tools/kern-tools-native_git.bb |    2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1

index ce94885..f2cd39f 100644
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
  DEPENDS = "git-native guilt-native"
 
-SRCREV = "a802ee9c8d9334c0f7932dfd40d45599addb7c90"
+SRCREV = "6f68c9473b43c3e39755a72aef8733cbd0bf1a59"
  PR = "r12"
  PV = "0.1+git${SRCPV}"
 

Quoted 7 lines · Darren Hart

Quoted 4 lines · Bruce Ashfield

Updating the SRCREV to import the following changes.

[updateme: find the board description with the highest score]

Quoted 7 lines

Quoted 4 lines

This removes the requirement that a custom linux-yocto .scc file have define KTYPE <foo>, where <foo> is typically "standard". The tools can now match on a .scc file that only matches the board, but will still chose one that matches the board and kernel type, if available.

Should the documentation then state that KTYPE is only necessary to define if it is not "standard" ? Does this also apply to

Nope. That isn't the intention. If you are using kernel types at all, you should define one. Whether it be standard or not.

Quoted 2 lines

linux-yocto-custom recipes/kernels where the repository could very well not define any ktype at all?

Same as the above. If you are create kernel .scc files to work with linux-yocto custom, you are now free to use kernel types, or not, your choice.

Quoted 29 lines

Quoted 17 lines

[updateme: allow for tabs or spaces in defines]

define KMACHINE<tab>$MACHINE was missed by the regex.

[scc/kgit-meta: detect and avoid duplicating patching]

To allow feature description to be included multiple times, they were previously split into -enable and 'patch' descriptions. With this change the patches will be detected as already included, and skipped automatically. Removing the need to do this split. It also cleans up the ability to warn about multiple includes.

[kconf_check: add "verify" configuration fragment type]

Quoted 7 lines

This adds the ability for a BSP to have a kernel configuration fragment that lists options that must be present. If they are not present it is a hard error. "required" is a similar fragment, but it adds them to the build, and audits them at the end, but does not abort the build if they are present. This is a minor distinction, but one that is useful when creating flexible, shared kernel config structures.

IIRC we discussed this verify thing at ELCE and how it broke some of the semantics.... trying to remember now, let's see:

Quoted 8 lines

kconf hardware foo.cfg
kconf verify hardware bar.cfg
kconf non-hardware foobar.cfg
kconf verify non-hardware barfoo.cfg

Is that how this is to be used? The configuration space just doubled from a documentation point of view, and that is without even considering the "required" keyword you described.

I'll continue to work on this for 1.4, but I didn't want to pend the series on anything that we discussed in Barcelona .. these have been around for some time and I wanted to push them out before doing any 1.4 tweaks.

But to answer your question. You could have multiple 'verify' fragments, but I'd only suggest one. It's a final check that critical options are in the final .config and will throw a hard error. required is an alias for 'hardware' and still only throws a warning if they are missing.

There few current users of 'verify', so I can still follow up with syntax tweaks that we discussed (do you have notes on that ?). I recall simply making 'verify' a modifier of the existing kconf types would be better than the current new type. I'll keep all the variants around, since the plumbing is the same and that will again give time for migration.

We could argue that required should also be a hard error, but we can't do that quite yet, since there are some existing use cases and trees that will start to error out, and I'd like to migrate them first.

Quoted 4 lines

Can you use required with verify? Can you use both of them with both hardware and non-hardware?

Any combination at all should work.

Cheers,

Bruce

Quoted 34 lines

Quoted 33 lines

[kconf_check: improve kernel audit report formatting] [kconf_check: perform validity checks on non-hardware options] [kconf_check: cleanups and verbose flag]

The existing output was verbose and not always useful to the reader. This change makes the output more compact, audits non-hardware options and gives information

[invalid (54)]: meta/cfg/preempt-rt/common-pc/invalid.cfg
This BSP sets config options that are not offered anywhere within this kernel

Signed-off-by Bruce Ashfield <[email protected]>

---
  meta/recipes-kernel/kern-tools/kern-tools-native_git.bb |    2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1

index ce94885..f2cd39f 100644
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
  DEPENDS = "git-native guilt-native"
 
-SRCREV = "a802ee9c8d9334c0f7932dfd40d45599addb7c90"
+SRCREV = "6f68c9473b43c3e39755a72aef8733cbd0bf1a59"
  PR = "r12"
  PV = "0.1+git${SRCPV}"
 

Quoted 22 lines · Bruce Ashfield

Quoted 7 lines · Darren Hart

Quoted 4 lines · Bruce Ashfield

Updating the SRCREV to import the following changes.

[updateme: find the board description with the highest score]

Quoted 7 lines

Quoted 4 lines

This removes the requirement that a custom linux-yocto .scc file have define KTYPE <foo>, where <foo> is typically "standard". The tools can now match on a .scc file that only matches the board, but will still chose one that matches the board and kernel type, if available.

Should the documentation then state that KTYPE is only necessary to define if it is not "standard" ? Does this also apply to

Nope. That isn't the intention. If you are using kernel types at all, you should define one. Whether it be standard or not.

Quoted 2 lines

linux-yocto-custom recipes/kernels where the repository could very well not define any ktype at all?

Quoted 3 lines

Same as the above. If you are create kernel .scc files to work with linux-yocto custom, you are now free to use kernel types, or not, your choice.

And what happens if they don't? It will just match on KMACHINE? Making it so you can define one defconfig per machine and be done with it? (if that's what you really wanted to do).

Quoted 56 lines

Quoted 31 lines

Quoted 19 lines

[updateme: allow for tabs or spaces in defines]

define KMACHINE<tab>$MACHINE was missed by the regex.

[scc/kgit-meta: detect and avoid duplicating patching]

To allow feature description to be included multiple times, they were previously split into -enable and 'patch' descriptions. With this change the patches will be detected as already included, and skipped automatically. Removing the need to do this split. It also cleans up the ability to warn about multiple includes.

[kconf_check: add "verify" configuration fragment type]

Quoted 7 lines

This adds the ability for a BSP to have a kernel configuration fragment that lists options that must be present. If they are not present it is a hard error. "required" is a similar fragment, but it adds them to the build, and audits them at the end, but does not abort the build if they are present. This is a minor distinction, but one that is useful when creating flexible, shared kernel config structures.

IIRC we discussed this verify thing at ELCE and how it broke some of the semantics.... trying to remember now, let's see:

Quoted 8 lines

kconf hardware foo.cfg
kconf verify hardware bar.cfg
kconf non-hardware foobar.cfg
kconf verify non-hardware barfoo.cfg

Is that how this is to be used? The configuration space just doubled from a documentation point of view, and that is without even considering the "required" keyword you described.

I'll continue to work on this for 1.4, but I didn't want to pend the series on anything that we discussed in Barcelona .. these have been around for some time and I wanted to push them out before doing any 1.4 tweaks.

But to answer your question. You could have multiple 'verify' fragments, but I'd only suggest one. It's a final check that critical options are in the final .config and will throw a hard error. required is an alias for 'hardware' and still only throws a warning if they are missing.

There few current users of 'verify', so I can still follow up with syntax tweaks that we discussed (do you have notes on that ?). I recall simply making 'verify' a modifier of the existing kconf types would be better than the current new type. I'll keep all the variants around, since the plumbing is the same and that will again give time for migration.

Quoted 3 lines

We could argue that required should also be a hard error, but we can't do that quite yet, since there are some existing use cases and trees that will start to error out, and I'd like to migrate them first.

I don't seem to have any notes on this for some reason. I'm wondering if instead of modifiers, these should just be kconf types:

non-hardware
hardware
required
verify

As you said, required is an alias to hardware, so it isn't really a modifier. And verify is less of a modifier and more of a verification step, so it doesn't matter if it's hardware or non-hardware.

This would reduce the config space from 6 to 4, not a huge savings, but it would be much simpler to document.

I'm going to omit these from the docs for now though.

Quoted 46 lines

Quoted 4 lines

Can you use required with verify? Can you use both of them with both hardware and non-hardware?

Any combination at all should work.

Cheers,

Bruce

Quoted 34 lines

Quoted 33 lines

[kconf_check: improve kernel audit report formatting] [kconf_check: perform validity checks on non-hardware options] [kconf_check: cleanups and verbose flag]

The existing output was verbose and not always useful to the reader. This change makes the output more compact, audits non-hardware options and gives information

[invalid (54)]: meta/cfg/preempt-rt/common-pc/invalid.cfg
This BSP sets config options that are not offered anywhere within this kernel

Signed-off-by Bruce Ashfield <[email protected]>

---
   meta/recipes-kernel/kern-tools/kern-tools-native_git.bb |    2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)

meta/recipes-kernel/kern-tools/kern-tools-native_git.bb +1 -1

index ce94885..f2cd39f 100644
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
   DEPENDS = "git-native guilt-native"
 
-SRCREV = "a802ee9c8d9334c0f7932dfd40d45599addb7c90"
+SRCREV = "6f68c9473b43c3e39755a72aef8733cbd0bf1a59"
   PR = "r12"
   PV = "0.1+git${SRCPV}"
 

Richard/Saul,

This is a collection of updates that I've had locally in the last few weeks leading up to the 1.3 release, but they were held back to avoid any negative impacts on the release.

I've kept all the kernel -stable and configuration updates as distinct commits, so it will be possible to separate any issues that crop up into manageable chunks. It also shows the different test points that I've used while updating the kernels.

For the kernel trees we get the following:

- the latest 3.2 and 3.4 -stable updates
- the latest 3.2 and 3.4 -rt updates
- configuration changes that have been collected for BSPs

** As far as I'm concerned these are all bugfix/stable changes and
can go into danny-next. Others may not agree :)

These have been build tested to the amount that my cycles allow, and others have already been used in the meta-intel BSPs. It's worth running this through a test build if you are so inclined, just to be sure.

For the tools we get the following updates:

- A fix for YOCTO #3226 that silences the warning:

find: warning: -path $oe-path/linux/ will not match anything because it ends with /.

** This change should be considering for the danny-next branch as well.

- A fix to the guilt SRC_URI to point at a valid tgz upstream.

** This change should be considering for the danny-next branch as well.

- kern-tools improvements and fixes that are part of the 1.4 feature work. The kern-tools commit has the details of the changes. They'll be further improved in the upcoming weeks, but they have been in use locally for over a month now, and have been tested by others as well, so they shouldn't cause any problems in master and are invisible to most users.

I've added Ross to the CC for the commits that can be pulled into the 1.3 maintenance branches.

I've added Rahul since there was some refactoring of the Rangely/dpdk configuration that should be checked as well.

Cheers,

Bruce

The following changes since commit 41125ddc8651494ffcf888d3b5efce1ae6d953bb:

   bitbake: print clear message for "bitbake -e ASSUME_PROVIDED" (2012-11-13 14:21:16 +0000)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib zedd/kernel
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (8):
linux-yocto/3.4: nfsd, pci, fishriver and rangely config changes linux-yocto/3.4: efi/mmc fixes and fri2 updates linux-yocto/3.4: bump kver to v3.4.16 linux-yocto/3.4: v3.4.17, v3.4.18, -rt and config changes linux-yocto/3.2: update to v3.2.32 and 3.2.32-rt48
kern-tools: kconf_check: fix find warning
kern-tools: flexibility and usability enhancements
guilt: change upstream tgz location

  meta/recipes-devtools/guilt/guilt-native_0.33.bb     |    4 ++--
  .../kern-tools/kern-tools-native_git.bb              |    2 +-
  meta/recipes-kernel/linux/linux-yocto-rt_3.2.bb      |    8 ++++----
  meta/recipes-kernel/linux/linux-yocto-rt_3.4.bb      |    8 ++++----
  meta/recipes-kernel/linux/linux-yocto-tiny_3.2.bb    |    6 +++---
  meta/recipes-kernel/linux/linux-yocto-tiny_3.4.bb    |    7 +++----
  meta/recipes-kernel/linux/linux-yocto_3.2.bb         |   18 +++++++++---------
  meta/recipes-kernel/linux/linux-yocto_3.4.bb         |   16 ++++++++--------
  8 files changed, 34 insertions(+), 35 deletions(-)