Kernel configuration check for recipes

Some user-space software has specific requirements about the kernel configuration options selected. This class allows a recipe to explicitly state the kernel configuration options it needs (through a REQUIRED_KERNEL_OPTIONS variable). This is a space separated list, where each item is one of:
(a) an option name (e.g. CONFIG_FHANDLE, in which case CONFIG_FHANDLE

must be set to y or m to match)
(b) optionname=value (e.g. CONFIG_FHANDLE=y, in which case CONFIG_FHANDLE must be set to y). If the specified value is n, it will also match if the option is not present.
(c) optionname1|optionname2|... (e.g. CONFIG_EXT2_FS|CONFIG_EXT4_USE_FOR_EXT23, meaning that either CONFIG_EXT2_FS or CONFIG_EXT4_USE_FOR_EXT23 must be set to y or m to match).

Inheriting the class will also add a dependency from do_configure on virtual/kernel:do_shared_workdir so that the kernel config is there to check. If one or more items are not matched, then a warning will be printed at do_configure time. (This is a warning rather than an error in case you are using linux-dummy with an externally built kernel).

A separate function is also provided should you wish to check a config option from python code - but note you must only call this in a place where you can guarantee that the kernel config has been written to the sysroot (i.e. from a task that has virtual/kernel:do_shared_workdir in its depends varflag value).

Fixes [YOCTO #5574].

Signed-off-by Paul Eggleton <[email protected]>

---
 meta/classes/kernel-check.bbclass | 97
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)
 create mode 100644 meta/classes/kernel-check.bbclass

meta/classes/kernel-check.bbclass new file +30 -0

new file mode 100644
index 0000000..d202f4e
@@ -0,0 +1,97 @@
+# Provides a means of checking within a recipe if particular kernel
+# config options are enabled
+#
+# Copyright (C) 2016 Intel Corporation
+#
+# Example usage (within a recipe):
+#
+# inherit kernel-check
+# REQUIRED_KERNEL_OPTIONS = "CONFIG_CGROUPS CONFIG_NAMESPACES"
+#
+# If one or more of the options aren't in the built kernel configuration
+# you will get a warning at do_configure time.
+#
+# You can also use the check_kernel_config_options() function to do
+# explicit checks yourself (and perform a different action).
+
+def check_kernel_config_options(options, d):
+    """
+    A function you can use to do explicit checks for kernel config
+    options from python code
+    """
+
+    if isinstance(options, basestring):
+        required = options.split()
+    else:
+        required = options[:]
+    missing = []
+    diffvalue = []
+    if required:
+        with open(d.expand('${STAGING_KERNEL_BUILDDIR}/.config'), 'r') as

f:

meta/classes/kernel-check.bbclass new file +18 -0

+            for line in f:
+                if line.startswith('#'):
+                    continue
+                linesplit = line.rstrip().split('=', 1)
+                if len(linesplit) < 2:
+                    continue
+                linevalue = linesplit[1]
+                for req in required:
+                    found = False
+                    if '|' in req:
+                        for reqitem in req.split('|'):
+                            if reqitem == linesplit[0]:
+                                if linevalue in ['y', 'm']:
+                                    found = True
+                                    break
+                    else:
+                        reqsplit = req.split('=', 1)
+                        # Can check for CONFIG_OPTION or

CONFIG_OPTION=value

meta/classes/kernel-check.bbclass new file +8 -0

+                        if len(reqsplit) > 1:
+                            reqvalue = reqsplit[1]
+                        else:
+                            reqvalue = None
+                        if reqsplit[0] == linesplit[0]:
+                            if reqvalue is None:
+                                if linevalue not in ['y', 'm']:
+                                    diffvalue.append((reqsplit[0], 'y or

m', linevalue))
+ elif reqvalue.strip("'\"") !=
linevalue.strip("'\""):
+ diffvalue.append((reqsplit[0], reqvalue,
linevalue))

meta/classes/kernel-check.bbclass new file +23 -0

+                            found = True
+
+                    if found:
+                        required.remove(req)
+                        break
+
+        for req in required:
+            reqsplit = req.split('=', 1)
+            if len(reqsplit) > 1:
+                if reqsplit[1] == 'n':
+                    continue
+                missing.append('%s=%s' % reqsplit)
+            else:
+                missing.append('%s' % req)
+
+    return missing, diffvalue
+
+
+python check_kernel_config() {
+    pn = d.getVar('PN', True)
+    required = d.getVar('REQUIRED_KERNEL_OPTIONS', True) or ''
+    if ' | ' in required:
+        bb.error('Invalid REQUIRED_KERNEL_OPTIONS value - cannot have

spaces around |')

meta/classes/kernel-check.bbclass new file +2 -0

+    if ' = ' in required:
+        bb.error('Invalid REQUIRED_KERNEL_OPTIONS value - cannot have

spaces around =')

meta/classes/kernel-check.bbclass new file +3 -0

+    missing, diffvalue = check_kernel_config_options(required, d)
+    if missing or diffvalue:
+        reqstr = '\n  '.join(missing + ['%s=%s (actual value %s)' % item

for item in diffvalue])
+ # Just warn here for cases like linux-dummy where we don't
actually

meta/classes/kernel-check.bbclass new file +2 -0

+        # know the final config
+        bb.warn('The kernel you are building against is missing the

following required configuration options:\n %s' % reqstr)

meta/classes/kernel-check.bbclass new file +6 -0

+}
+
+python() {
+    if d.getVar('REQUIRED_KERNEL_OPTIONS', True):
+        d.appendVar('DEPENDS', ' virtual/kernel')
+        d.appendVarFlag('do_configure', 'prefuncs', '

check_kernel_config')
+ d.appendVarFlag('do_configure', 'depends', '
virtual/kernel:do_shared_workdir')
+}
--
2.5.5

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

A lot of the IMAGE_FSTYPES items require the appropriate filesystem driver to be enabled in the kernel configuration; e.g. in order to read a btrfs filesystem, the kernel must enable CONFIG_BTRFS_FS. Add a check to ensure that is the case.

Fixes [YOCTO #5574].

Signed-off-by Paul Eggleton <[email protected]>

---
 meta/classes/image.bbclass       | 23 +++++++++++++++++++++++
 meta/classes/image_types.bbclass | 11 +++++++++++
 2 files changed, 34 insertions(+)

meta/classes/image.bbclass +5 -0

index 4542e95..c56b053 100644
@@ -238,9 +238,32 @@ do_rootfs[cleandirs] += "${S}"
 do_rootfs[umask] = "022"
 addtask rootfs before do_build
 
+inherit kernel-check
+
+def check_image_fstypes_kernel(d):
+    """
+    Check that the kernel we have built has the appropriate config

options enabled

meta/classes/image.bbclass +6 -0

+    to support the image formats specified in IMAGE_FSTYPES
+    """
+    fstypes = (d.getVar('IMAGE_FSTYPES', True) or '').split()
+    ctypes = (d.getVar('COMPRESSIONTYPES', True) or '').split()
+    for fstype in fstypes:
+        kernconfig = (d.getVar('IMAGE_TYPE_KERNEL_OPTIONS_' + fstype,

True) or '').split()

meta/classes/image.bbclass +8 -0

+        for ctype in ctypes:
+            if fstype.endswith("." + ctype):
+                basetype = fstype[:-len("." + ctype)]
+                kernconfig.extend((d.getVar('IMAGE_TYPE_KERNEL_OPTIONS_'
+ basetype, True) or '').split())
+        kernconfig = list(set(kernconfig))
+        if kernconfig:
+            missing, diffvalue = check_kernel_config_options(kernconfig,

d)

meta/classes/image.bbclass +2 -0

+            if missing or diffvalue:
+                bb.warn('IMAGE_FSTYPES contains %s, but the following

required kernel configuration items are not present in the kernel configuration:\n %s' % (fstype, '\n '.join(missing + ['%s=%s (actual value %s)' % item for item in diffvalue])))

meta/classes/image.bbclass +3 -0

+
 fakeroot python do_image () {
     from oe.utils import execute_pre_post_process
 
+    check_image_fstypes_kernel(d)
+
     pre_process_cmds = d.getVar("IMAGE_PREPROCESS_COMMAND", True)
 
     execute_pre_post_process(d, pre_process_cmds)

meta/classes/image_types.bbclass +4 -0

index 53af7ca..dd79726 100644
@@ -240,6 +240,17 @@ IMAGE_DEPENDS_ubifs = "mtd-utils-native"
 IMAGE_DEPENDS_multiubi = "mtd-utils-native"
 IMAGE_DEPENDS_wic = "parted-native"
 
+IMAGE_TYPE_KERNEL_OPTIONS_jffs2 = "CONFIG_JFFS2_FS"
+IMAGE_TYPE_KERNEL_OPTIONS_jffs2.sum = "CONFIG_JFFS2_SUMMARY"
+IMAGE_TYPE_KERNEL_OPTIONS_cramfs = "CONFIG_CRAMFS"
+IMAGE_TYPE_KERNEL_OPTIONS_ext2 =

"CONFIG_EXT2_FS|CONFIG_EXT4_USE_FOR_EXT23"
+IMAGE_TYPE_KERNEL_OPTIONS_ext3 =
"CONFIG_EXT3_FS|CONFIG_EXT4_USE_FOR_EXT23"

meta/classes/image_types.bbclass +6 -0

+IMAGE_TYPE_KERNEL_OPTIONS_ext4 = "CONFIG_EXT4_FS"
+IMAGE_TYPE_KERNEL_OPTIONS_btrfs = "CONFIG_BTRFS_FS"
+IMAGE_TYPE_KERNEL_OPTIONS_squashfs = "CONFIG_SQUASHFS"
+IMAGE_TYPE_KERNEL_OPTIONS_squashfs-xz = "CONFIG_SQUASHFS_XZ"
+IMAGE_TYPE_KERNEL_OPTIONS_squashfs-lzo = "CONFIG_SQUASHFS_LZO"
+
 # This variable is available to request which values are suitable for

IMAGE_FSTYPES

 IMAGE_TYPES = " \
     jffs2 jffs2.sum \

--
2.5.5

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Add the capability for other recipes to verify that kernel configuration options they depend upon at runtime are present, and warn if they are missing. This seems functional enough to me but is still in RFC - please let me know if it looks OK to you or I've missed anything (no doubt there are recipes we could add the checks to, I refer mainly to the mechanism itself).

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit
ece101be5158beee709cdfbb85ecdbdc8d9fb864:

test-empty-image: Fix LIC_FILES_CHKSUM typo (2016-05-06 10:47:59 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/kernel-check

http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/kernel-check

Paul Eggleton (6):
classes/kernel-check: add a class to check kernel config options
eudev: check for required kernel config options
systemd: check for required kernel config options classes/kernel: fix typo
classes/kernel: check OLDEST_KERNEL at configure time classes/image: check kernel config supports IMAGE_FSTYPES items

 meta/classes/image.bbclass               | 23 ++++++++
 meta/classes/image_types.bbclass         | 11 ++++
 meta/classes/kernel-check.bbclass        | 97
++++++++++++++++++++++++++++++++
 meta/classes/kernel.bbclass              | 16 +++++-
 meta/recipes-core/systemd/systemd_229.bb | 19 ++++++-
 meta/recipes-core/udev/eudev_3.1.5.bb    | 22 +++++++-
 6 files changed, 185 insertions(+), 3 deletions(-)
 create mode 100644 meta/classes/kernel-check.bbclass

--
2.5.5

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Quoted 5 lines · Paul Eggleton

A lot of the IMAGE_FSTYPES items require the appropriate filesystem driver to be enabled in the kernel configuration; e.g. in order to read a btrfs filesystem, the kernel must enable CONFIG_BTRFS_FS. Add a check to ensure that is the case.

So what's the overall design for this ? Is it documented anywhere ? This is something
that we talked about a couple of years and I have reservations and objections to
the current implementation.

.. but since I can only see bits and chunks of patches, I'd rather read something
complete and offer some more detailed feedback.

Bottom line: I see a slippery slope to tightly specified option that depend on kernel
versions, I see kernel options sprayed all over layers, etc, etc. Which is exactly
what we've been trying to avoid with the centralized kernel meta-data, and again,
there's a mechanism that I finished early this year to extend those options to any
kernel, enforce, warn, error, etc .. but now, I'll just toss it into the bin.

Quoted 20 lines · Bruce Ashfield

Quoted 5 lines · Paul Eggleton

A lot of the IMAGE_FSTYPES items require the appropriate filesystem driver to be enabled in the kernel configuration; e.g. in order to read a btrfs filesystem, the kernel must enable CONFIG_BTRFS_FS. Add a check to ensure that is the case.

So what's the overall design for this ? Is it documented anywhere ? This is something
that we talked about a couple of years and I have reservations and objections to
the current implementation.

.. but since I can only see bits and chunks of patches, I'd rather read something
complete and offer some more detailed feedback.

Quoted 11 lines

Bottom line: I see a slippery slope to tightly specified option that depend on kernel
versions, I see kernel options sprayed all over layers, etc, etc. Which is exactly
what we've been trying to avoid with the centralized kernel meta-data, and again,
there's a mechanism that I finished early this year to extend those options to any
kernel, enforce, warn, error, etc .. but now, I'll just toss it into the bin.

Perhaps this functionality should be by feature or capability rather than individual options, and let the kernel build determine what features it provides based on its configuration.

--
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Quoted 4 lines · Paul Eggleton

A lot of the IMAGE_FSTYPES items require the appropriate filesystem driver to be enabled in the kernel configuration; e.g. in order to read a btrfs filesystem, the kernel must enable CONFIG_BTRFS_FS. Add a check to ensure that is the case.

So what's the overall design for this ? Is it documented anywhere ? This is something that we talked about a couple of years and I have reservations and objections to the current implementation.

.. but since I can only see bits and chunks of patches, I'd rather read something complete and offer some more detailed feedback.

Bottom line: I see a slippery slope to tightly specified option that depend on kernel versions, I see kernel options sprayed all over layers, etc, etc. Which is exactly what we've been trying to avoid with the centralized kernel meta-data, and again, there's a mechanism that I finished early this year to extend those options to any kernel, enforce, warn, error, etc .. but now, I'll just toss it into the bin.

Quoted 17 lines · Paul Eggleton

Some user-space software has specific requirements about the kernel configuration options selected. This class allows a recipe to explicitly state the kernel configuration options it needs (through a REQUIRED_KERNEL_OPTIONS variable). This is a space separated list, where

each item is one of:
(a) an option name (e.g. CONFIG_FHANDLE, in which case CONFIG_FHANDLE

must be set to y or m to match)

(b) optionname=value (e.g. CONFIG_FHANDLE=y, in which case

CONFIG_FHANDLE must be set to y). If the specified value is n, it will also match if the option is not present.

(c) optionname1|optionname2|... (e.g.

Quoted 3 lines

CONFIG_EXT2_FS|CONFIG_EXT4_USE_FOR_EXT23, meaning that either CONFIG_EXT2_FS or CONFIG_EXT4_USE_FOR_EXT23 must be set to y or m to match).

This duplicates code that is already in the kconf_check routines, which I've already made generic (and added dependency information) for the 2.2 release ... so call me confused that I've never heard about this until now :(

On Mon, May 9, 2016 at 12:43 AM, Paul Eggleton <

Quoted 8 lines · [email protected]> wrote:

Add the capability for other recipes to verify that kernel configuration options they depend upon at runtime are present, and warn if they are missing. This seems functional enough to me but is still in RFC - please let me know if it looks OK to you or I've missed anything (no doubt there are recipes we could add the checks to, I refer mainly to the mechanism itself).

Quoted 5 lines

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

See my replies to the patches. I'm sure it won't matter in the end, but this is a duplication of the mechanisms that we already have in the kernel config check phase. I have a whole set of changes that I completed early this year that extends that mechanism to any kernel (not just kernel-yocto) classes. In that mechanism there's the concept of "required" and "optional" Kconfig options. If they aren't present in the final .config, we warn and optionally error (this is versus the currently mechanism of only warning if boot critical options are dropped).

I'm betting that the follow on to this series will be "Hey recipes should set the options for the kernel if they really need them" .. which IMHO is an incredibly bad idea.

wrote:

Hi Bruce,

Quoted 8 lines · On Mon, 09 May 2016 08:05:06 Bruce Ashfield wrote:

Quoted 4 lines · Paul Eggleton

A lot of the IMAGE_FSTYPES items require the appropriate filesystem driver to be enabled in the kernel configuration; e.g. in order to read a btrfs filesystem, the kernel must enable CONFIG_BTRFS_FS. Add a check to ensure that is the case.

So what's the overall design for this ? Is it documented anywhere ? This

is

Quoted 1 line

something that we talked about a couple of years and I have reservations

and

Quoted 6 lines

objections to the current implementation.

.. but since I can only see bits and chunks of patches, I'd rather read something complete and offer some more detailed feedback.

Bottom line: I see a slippery slope to tightly specified option that

depend

Quoted 1 line

on kernel versions, I see kernel options sprayed all over layers, etc,

etc.

Quoted 1 line

Which is exactly what we've been trying to avoid with the centralized

kernel

Quoted 1 line

meta-data, and again, there's a mechanism that I finished early this

year to

Quoted 2 lines

extend those options to any kernel, enforce, warn, error, etc .. but now, I'll just toss it into the bin.

There's no design written out beyond what you see in the associated bug:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=5574

The situation I am trying to avoid is the exact one mentioned by a commenter
on that bug - which is that if I build my rootfs in a particular way, and couple it with a particular kernel configuration, I want the build system to
tell me if those are going to be incompatible. The current situation where the
you don't find out until the system doesn't boot (and you may not get any meaningful error message at that point anyway) isn't great.

You mention hardware in your reply to Chris, but this mostly isn't about hardware - it's about dependencies on software features of the kernel from

userspace. I'm perfectly happy if we go for alternative solution, but to my mind whatever solution we come up with needs to be something whereby if you enable something in userspace that the kernel as currently configured isn't going to be able to support, you at least get a warning at build time.

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre

wrote:

Quoted 4 lines · On Mon, 09 May 2016 07:59:43 Bruce Ashfield wrote:

Quoted 2 lines · Paul Eggleton

Some user-space software has specific requirements about the kernel configuration options selected. This class allows a recipe to

explicitly

Quoted 2 lines

Quoted 2 lines

state the kernel configuration options it needs (through a REQUIRED_KERNEL_OPTIONS variable). This is a space separated list,

where

Quoted 13 lines

Quoted 13 lines

each item is one of:
(a) an option name (e.g. CONFIG_FHANDLE, in which case CONFIG_FHANDLE

must be set to y or m to match)

(b) optionname=value (e.g. CONFIG_FHANDLE=y, in which case

CONFIG_FHANDLE must be set to y). If the specified value is n, it will also match if the option is not present.

(c) optionname1|optionname2|... (e.g.

Quoted 7 lines

Quoted 3 lines

CONFIG_EXT2_FS|CONFIG_EXT4_USE_FOR_EXT23, meaning that either CONFIG_EXT2_FS or CONFIG_EXT4_USE_FOR_EXT23 must be set to y or m to match).

This duplicates code that is already in the kconf_check routines, which I've already made generic (and added dependency information) for the 2.2 release ... so call me confused that I've never heard about this until

now

Quoted 1 line

:(

I guess I didn't look hard enough, I'll have to take a look at that. This is
why the series is RFC (and I included you on CC ;)

Cheers,
Paul

--

Paul Eggleton
Intel Open Source Technology Centre

Use the list in the systemd 225 README to set required Linux kernel config options.

Fixes [YOCTO #5574].

Signed-off-by Paul Eggleton <[email protected]>

---
meta/recipes-core/systemd/systemd_229.bb | 19 ++++++++++++++++++-

1 file changed, 18 insertions(+), 1 deletion(-)

meta/recipes-core/systemd/systemd_229.bb +0 -0

index c23c749..d64b3cd 100644
@@ -22,7 +22,7 @@ DEPENDS = "kmod docbook-sgml-dtd-4.1-native intltool-native gperf-native acl rea
 

SECTION = "base/shell"

meta/recipes-core/systemd/systemd_229.bb +1 -1

-inherit useradd pkgconfig autotools perlnative update-rc.d update-alternatives qemu systemd ptest gettext bash-completion
+inherit useradd pkgconfig autotools perlnative update-rc.d update-alternatives qemu systemd ptest gettext bash-completion kernel-check

SRCREV = "714c62b46379abb7558c544665522aca91691e10"

meta/recipes-core/systemd/systemd_229.bb +0 -0

@@ -192,6 +192,23 @@ CFLAGS .= "${@bb.utils.contains('PACKAGECONFIG', 'valgrind', ' -DVALGRIND=1', ''
 # disable problematic GCC 5.2 optimizations [YOCTO #8291]

FULL_OPTIMIZATION_append_arm = " -fno-schedule-insns -fno-schedule-insns2"

meta/recipes-core/systemd/systemd_229.bb +5 -0

+# All the required options from the systemd README
+REQUIRED_KERNEL_OPTIONS = "\
+                           CONFIG_DEVTMPFS \
+                           CONFIG_CGROUPS \
+                           CONFIG_INOTIFY_USER \

meta/recipes-core/systemd/systemd_229.bb +2 -0

+                           "
+

do_configure_prepend() {
	export NM="${HOST_PREFIX}gcc-nm"
	export AR="${HOST_PREFIX}gcc-ar"

--
2.5.5

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

If the kernel being built is older than OLDEST_KERNEL and we're building with glibc, then the C library we're building is probably not going to be compatible with the kernel and we should warn the user. (This is easier to do here rather than when building glibc, because we don't necessarily have the information we need to determine the kernel version there, whereas we do here.)

Fixes [YOCTO #8653].

Signed-off-by Paul Eggleton <[email protected]>

---
meta/classes/kernel.bbclass | 14 ++++++++++++++

1 file changed, 14 insertions(+)

meta/classes/kernel.bbclass +0 -0

index fa0864c..af88cc0 100644
@@ -330,6 +330,20 @@ sysroot_stage_all () {
 

KERNEL_CONFIG_COMMAND ?= "oe_runmake_call -C ${S} O=${B} oldnoconfig || yes '' | oe_runmake -C ${S} O=${B} oldconfig"

meta/classes/kernel.bbclass +4 -0

+
+check_oldest_kernel[vardepsexclude] += "OLDEST_KERNEL KERNEL_VERSION"
+do_configure[prefuncs] += "check_oldest_kernel"
+

kernel_do_configure() {
	# fixes extra + in /lib/modules/2.6.37+
	# $ scripts/setlocalversion . => +

--
2.5.5

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core