Skip to contentSkip to conversation

kernel-module-split: Allow for external conf files

PATCH Superseded Michal Sieron 2 participants 6 messages Apr 10, 2025 – May 15, 2025

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kernel-module-split: Allow for external conf files

Some recipes might provide conf files produced during build phase or simply tracked in the VCS instead of generating them with Yocto. In such cases those conf files wouldn't be assigned to correct packages. With this change, if user wants to generate a conf file they still can, but not generating them won't prevent assigning the file to proper package given the file exists.

---
 .../kernel-module-split.bbclass               | 25 +++++++++++--------
 1 file changed, 14 insertions(+), 11 deletions(-)
meta/classes-recipe/kernel-module-split.bbclass +14 -11
index 9487365eb7..6da3cb7fc7 100644
@@ -99,9 +99,9 @@ python split_kernel_module_packages () {
             bb.warn("module_autoload_%s was replaced by KERNEL_MODULE_AUTOLOAD for cases where basename == module name, please drop it" % basename)
         if autoload and basename not in autoloadlist:
             bb.warn("module_autoload_%s is defined but '%s' isn't included in KERNEL_MODULE_AUTOLOAD, please add it there" % (basename, basename))
+        conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
+        name = '%s%s' % (dvar, conf)
         if basename in autoloadlist:
-            conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 if autoload:
@@ -109,31 +109,34 @@ python split_kernel_module_packages () {
                         f.write('%s\n' % m)
                 else:
                     f.write('%s\n' % basename)
-            conf2append = ' %s' % conf
-            d.appendVar('FILES:%s' % pkg, conf2append)
-            d.appendVar('CONFFILES:%s' % pkg, conf2append)
             postinst = d.getVar('pkg_postinst:%s' % pkg)
             if not postinst:
                 bb.fatal("pkg_postinst:%s not defined" % pkg)
             postinst += d.getVar('autoload_postinst_fragment') % (autoload or basename)
             d.setVar('pkg_postinst:%s' % pkg, postinst)
 
+        if os.path.exists(name):
+            conf2append = ' %s' % conf
+            d.appendVar('FILES:%s' % pkg, conf2append)
+            d.appendVar('CONFFILES:%s' % pkg, conf2append)
+
         # Write out any modconf fragment
         modconflist = (d.getVar("KERNEL_MODULE_PROBECONF") or "").split()
         modconf = d.getVar('module_conf_%s' % basename)
+        conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
+        name = '%s%s' % (dvar, conf)
         if modconf and basename in modconflist:
-            conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 f.write("%s\n" % modconf)
-            conf2append = ' %s' % conf
-            d.appendVar('FILES:%s' % pkg, conf2append)
-            d.appendVar('CONFFILES:%s' % pkg, conf2append)
-
         elif modconf:
             bb.error("Please ensure module %s is listed in KERNEL_MODULE_PROBECONF since module_conf_%s is set" % (basename, basename))
 
+        if os.path.exists(name):
+            conf2append = ' %s' % conf
+            d.appendVar('FILES:%s' % pkg, conf2append)
+            d.appendVar('CONFFILES:%s' % pkg, conf2append)
+
         if "description" in vals:
             old_desc = d.getVar('DESCRIPTION:' + pkg) or ""
             d.setVar('DESCRIPTION:' + pkg, old_desc + "; " + vals["description"])
Signature · 1 line
2.49.0
#1 lore raw
Quoted 7 lines · Michal Sieron via lists.openembedded.org

Some recipes might provide conf files produced during build phase or simply tracked in the VCS instead of generating them with Yocto. In such cases those conf files wouldn't be assigned to correct packages. With this change, if user wants to generate a conf file they still can, but not generating them won't prevent assigning the file to proper package given the file exists.

Whenever we add a different code path, we need to add a test to the oe selftests, both for the existing case (if there isn't one already) and the new one. Without those tests, we'll get a better understanding of how this is supposed to work, that it doesn't break existing users and we'll know when it breaks in the future (since this obviously won't be a default path with our reference distros).

Quoted 19 lines
---
 .../kernel-module-split.bbclass               | 25 +++++++++++--------
 1 file changed, 14 insertions(+), 11 deletions(-)
meta/classes-recipe/kernel-module-split.bbclass +0 -0
index 9487365eb7..6da3cb7fc7 100644
@@ -99,9 +99,9 @@ python split_kernel_module_packages () {
             bb.warn("module_autoload_%s was replaced by

KERNEL_MODULE_AUTOLOAD for cases where basename == module name, please drop it" % basename)
if autoload and basename not in autoloadlist:
bb.warn("module_autoload_%s is defined but '%s' isn't
included in KERNEL_MODULE_AUTOLOAD, please add it there" % (basename, basename))

meta/classes-recipe/kernel-module-split.bbclass python split_kernel_module_packages () { · line 104

Quoted 5 lines
meta/classes-recipe/kernel-module-split.bbclass +2 -2
+        conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
+        name = '%s%s' % (dvar, conf)
         if basename in autoloadlist:
-            conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
-            name = '%s%s' % (dvar, conf)

A comment about how your high level description in the long log relates to needing to move these variables outside of the conditional would be helpful.

Quoted 12 lines

os.makedirs(os.path.dirname(name), exist_ok=True) with open(name, 'w') as f:
if autoload:

meta/classes-recipe/kernel-module-split.bbclass +0 -3
@@ -109,31 +109,34 @@ python split_kernel_module_packages () {
                         f.write('%s\n' % m)
                 else:
                     f.write('%s\n' % basename)
-            conf2append = ' %s' % conf
-            d.appendVar('FILES:%s' % pkg, conf2append)
-            d.appendVar('CONFFILES:%s' % pkg, conf2append)
             postinst = d.getVar('pkg_postinst:%s' % pkg)
             if not postinst:

meta/classes-recipe/kernel-module-split.bbclass python split_kernel_module_packages () { · line 123

Quoted 10 lines
meta/classes-recipe/kernel-module-split.bbclass +4 -0
                 bb.fatal("pkg_postinst:%s not defined" % pkg)
             postinst += d.getVar('autoload_postinst_fragment') %
 (autoload or basename)
             d.setVar('pkg_postinst:%s' % pkg, postinst)
 
+        if os.path.exists(name):
+            conf2append = ' %s' % conf
+            d.appendVar('FILES:%s' % pkg, conf2append)
+            d.appendVar('CONFFILES:%s' % pkg, conf2append)
 

A quick comment here would help as well. This is now outside the conditional, so a comment indicating that the file at "name" can either be generated above or placed there by a recipe will help with maintenance.

I'm thinking that we should also log when a supplied configuration file is used. Otherwise, it may not be obvious if a runtime issue is caused.

Also, what happens if the file at "name" is updated ? I don't think the module splitting would be re-run, so would we have a "stale" file ? Should that file be something in the kernel-module recipe that triggers a rebuild if it is changed ? I don't see that mentioned (maybe I'm imagining the problem) or a requirement on the recipe.

Bruce

Quoted 44 lines
meta/classes-recipe/kernel-module-split.bbclass +8 -6
+
         # Write out any modconf fragment
         modconflist = (d.getVar("KERNEL_MODULE_PROBECONF") or "").split()
         modconf = d.getVar('module_conf_%s' % basename)
+        conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
+        name = '%s%s' % (dvar, conf)
         if modconf and basename in modconflist:
-            conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 f.write("%s\n" % modconf)
-            conf2append = ' %s' % conf
-            d.appendVar('FILES:%s' % pkg, conf2append)
-            d.appendVar('CONFFILES:%s' % pkg, conf2append)
-
         elif modconf:
             bb.error("Please ensure module %s is listed in
 KERNEL_MODULE_PROBECONF since module_conf_%s is set" % (basename, basename))
 
+        if os.path.exists(name):
+            conf2append = ' %s' % conf
+            d.appendVar('FILES:%s' % pkg, conf2append)
+            d.appendVar('CONFFILES:%s' % pkg, conf2append)
+

if "description" in vals:
old_desc = d.getVar('DESCRIPTION:' + pkg) or "" d.setVar('DESCRIPTION:' + pkg, old_desc + "; " +
vals["description"])
--
2.49.0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#214633):
https://lists.openembedded.org/g/openembedded-core/message/214633
Mute This Topic: https://lists.openembedded.org/mt/112187964/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II
#2 lore raw
Quoted 6 lines

Whenever we add a different code path, we need to add a test to the oe selftests, both for the existing case (if there isn't one already) and the new one. Without those tests, we'll get a better understanding of how this is supposed to work, that it doesn't break existing users and we'll know when it breaks in the future (since this obviously won't be a default path with our reference distros).

The problem with this is that there are no existing tests for kernel-module-split.bbclass. So I would need some heavy assistance with writing such test cases :(

But I do agree that tests are generally a good idea.
The whole reason I sent the patch is because I noticed that behavior change between kirkstone and scarthgap caused by this change in oe-core:71460993f350bca3d5a22115fd5551696f955c9f.

Quoted 3 lines

A comment about how your high level description in the long log relates to needing to move these variables outside of the conditional would be helpful.

...

Quoted 4 lines

A quick comment here would help as well. This is now outside the conditional, so a comment indicating that the file at "name" can either be generated above or placed there by a recipe will help with maintenance.

Will try to somehow explain it in v2 of the patch.

Quoted 6 lines

Also, what happens if the file at "name" is updated ? I don't think the module splitting would be re-run, so would we have a "stale" file ? Should that file be something in the kernel-module recipe that triggers a rebuild if it is changed ? I don't see that mentioned (maybe I'm imagining the problem) or a requirement on the recipe.

Wouldn't that alter do_install's checksum? Otherwise I don't think I have an answer.

Anyway, while I was preparing this response I noticed some other things.

1. pkg_postinst hook
There is this pkg_postinst part for "autoload". I guess I should move that hook so it is also installed when .conf is vendored, is that right? Although that wasn't there before the regression commit I found.

2. Duplicated entries in (CONF)FILES:*
Even before my patch FILES:* and CONFFILES:* entries for kernel module packages are for some reason doubled. But that is probably for another patch.

Best regards,
Michal

#3 lore raw

In message: Re: [OE-core] [PATCH] kernel-module-split: Allow for external conf files

Quoted 10 lines · on 17/04/2025 Michal Sieron wrote:
Quoted 6 lines

Whenever we add a different code path, we need to add a test to the oe selftests, both for the existing case (if there isn't one already) and the new one. Without those tests, we'll get a better understanding of how this is supposed to work, that it doesn't break existing users and we'll know when it breaks in the future (since this obviously won't be a default path with our reference distros).

The problem with this is that there are no existing tests for kernel-module-split.bbclass. So I would need some heavy assistance with writing such test cases :(

We can definitely help with this, having specific questions will help of course, but this is worth the effort (and in fact may be the only way to get this merged).

I'm no expert in self-tests either, but can offer some of my time to sort through the issue.

The code is simply to complex now to not be tested at all. We don't need something complex, just something that ensures a conf file is generated in both cases.

Quoted 5 lines

But I do agree that tests are generally a good idea.
The whole reason I sent the patch is because I noticed that behavior change between kirkstone and scarthgap caused by this change in oe-core:71460993f350bca3d5a22115fd5551696f955c9f.

Agreed!

Quoted 22 lines
Quoted 4 lines

A comment about how your high level description in the long log relates to needing to move these variables outside of the conditional would be helpful.

...

Quoted 4 lines

A quick comment here would help as well. This is now outside the conditional, so a comment indicating that the file at "name" can either be generated above or placed there by a recipe will help with maintenance.

Will try to somehow explain it in v2 of the patch.

Quoted 6 lines

Also, what happens if the file at "name" is updated ? I don't think the module splitting would be re-run, so would we have a "stale" file ? Should that file be something in the kernel-module recipe that triggers a rebuild if it is changed ? I don't see that mentioned (maybe I'm imagining the problem) or a requirement on the recipe.

Wouldn't that alter do_install's checksum? Otherwise I don't think I have an answer.

Quoted 6 lines

Anyway, while I was preparing this response I noticed some other things.

1. pkg_postinst hook
There is this pkg_postinst part for "autoload". I guess I should move that hook so it is also installed when .conf is vendored, is that right? Although that wasn't there before the regression commit I found.

At a glance .. yes, we'd want to keep that same behavior in both modes.

Quoted 5 lines

2. Duplicated entries in (CONF)FILES:*
Even before my patch FILES:* and CONFFILES:* entries for kernel module packages are for some reason doubled. But that is probably for another patch.

Yes, it would be best to do this patch, and a second separate one that sorts that out. If the logical changes are separate, they are easier to review and merge.

Bruce

Quoted 3 lines

Best regards,
Michal

#4 lore raw

PATCH v2 1/1 kernel-module-split: Allow for external conf files

Some recipes might provide conf files produced during build phase or simply tracked in the VCS instead of generating them with Yocto. In such cases those conf files wouldn't be assigned to correct packages. With this change, if user wants to generate a conf file they still can, but not generating them won't prevent assigning the file to proper package given the file exists.

---

I am quite busy lately so didn't have time to go back to this topic.

This v2 differs from v1 by added comments and post install hook is now treated the same way as (CONF)FILES:* variable modification.

 .../kernel-module-split.bbclass               | 27 +++++++++++++------
 1 file changed, 19 insertions(+), 8 deletions(-)
meta/classes-recipe/kernel-module-split.bbclass +19 -8
index 9487365eb7..a2d81f18e2 100644
@@ -99,9 +99,12 @@ python split_kernel_module_packages () {
             bb.warn("module_autoload_%s was replaced by KERNEL_MODULE_AUTOLOAD for cases where basename == module name, please drop it" % basename)
         if autoload and basename not in autoloadlist:
             bb.warn("module_autoload_%s is defined but '%s' isn't included in KERNEL_MODULE_AUTOLOAD, please add it there" % (basename, basename))
+
+        # The .conf file can either be installed by a recipe or generated from module_autoload_*
+        conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
+        name = '%s%s' % (dvar, conf)
+        # If module name is in KERNEL_MODULE_AUTOLOAD, then generate the .conf file and write to `name`.
         if basename in autoloadlist:
-            conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 if autoload:
@@ -109,6 +112,9 @@ python split_kernel_module_packages () {
                         f.write('%s\n' % m)
                 else:
                     f.write('%s\n' % basename)
+        # If the .conf file exits, then add it to FILES:* and CONFFILES:* and add postinstall hook.
+        # It doesn't matter if it was generated from module_autoload_* or installed by the recipe.
+        if os.path.exists(name):
             conf2append = ' %s' % conf
             d.appendVar('FILES:%s' % pkg, conf2append)
             d.appendVar('CONFFILES:%s' % pkg, conf2append)
@@ -121,18 +127,23 @@ python split_kernel_module_packages () {
         # Write out any modconf fragment
         modconflist = (d.getVar("KERNEL_MODULE_PROBECONF") or "").split()
         modconf = d.getVar('module_conf_%s' % basename)
+
+        # The .conf file can either be installed by a recipe or generated from module_conf_*
+        conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
+        name = '%s%s' % (dvar, conf)
+        # If module name is in KERNEL_MODULE_PROBECONF, then generate the .conf file and write to `name`.
         if modconf and basename in modconflist:
-            conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 f.write("%s\n" % modconf)
-            conf2append = ' %s' % conf
-            d.appendVar('FILES:%s' % pkg, conf2append)
-            d.appendVar('CONFFILES:%s' % pkg, conf2append)
-
         elif modconf:
             bb.error("Please ensure module %s is listed in KERNEL_MODULE_PROBECONF since module_conf_%s is set" % (basename, basename))
+        # If the .conf file exits, then add it to FILES:* and CONFFILES:*.
+        # It doesn't matter if it was generated from module_conf_* or installed by the recipe.
+        if os.path.exists(name):
+            conf2append = ' %s' % conf
+            d.appendVar('FILES:%s' % pkg, conf2append)
+            d.appendVar('CONFFILES:%s' % pkg, conf2append)
 
         if "description" in vals:
             old_desc = d.getVar('DESCRIPTION:' + pkg) or ""
Signature · 1 line
2.49.0
#5 lore raw
Quoted 7 lines · Michal Sieron

Some recipes might provide conf files produced during build phase or simply tracked in the VCS instead of generating them with Yocto. In such cases those conf files wouldn't be assigned to correct packages. With this change, if user wants to generate a conf file they still can, but not generating them won't prevent assigning the file to proper package given the file exists.

Quoted 8 lines

---

I am quite busy lately so didn't have time to go back to this topic.

This v2 differs from v1 by added comments and post install hook is now treated the same way as (CONF)FILES:* variable modification.

And sorry for my very slow reply.

This version of the patch was hidden below the original, so I missed it when it arrived.

Quoted 34 lines
 .../kernel-module-split.bbclass               | 27 +++++++++++++------
 1 file changed, 19 insertions(+), 8 deletions(-)
meta/classes-recipe/kernel-module-split.bbclass +0 -0
index 9487365eb7..a2d81f18e2 100644
@@ -99,9 +99,12 @@ python split_kernel_module_packages () {
             bb.warn("module_autoload_%s was replaced by

KERNEL_MODULE_AUTOLOAD for cases where basename == module name, please drop it" % basename)
if autoload and basename not in autoloadlist:
bb.warn("module_autoload_%s is defined but '%s' isn't
included in KERNEL_MODULE_AUTOLOAD, please add it there" % (basename, basename))

meta/classes-recipe/kernel-module-split.bbclass +2 -0
+
+        # The .conf file can either be installed by a recipe or generated

from module_autoload_*

meta/classes-recipe/kernel-module-split.bbclass +3 -0
+        conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
+        name = '%s%s' % (dvar, conf)
+        # If module name is in KERNEL_MODULE_AUTOLOAD, then generate the

.conf file and write to name.
if basename in autoloadlist:

meta/classes-recipe/kernel-module-split.bbclass +1 -2
-            conf = '%s/%s.conf' % (d.getVar('modulesloaddir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 if autoload:
@@ -109,6 +112,9 @@ python split_kernel_module_packages () {
                         f.write('%s\n' % m)
                 else:
                     f.write('%s\n' % basename)
+        # If the .conf file exits, then add it to FILES:* and CONFFILES:*
Quoted 2 lines

and add postinstall hook.

Minor typo s/exits/exists/

Quoted 35 lines

+ # It doesn't matter if it was generated from module_autoload_* or
installed by the recipe.

meta/classes-recipe/kernel-module-split.bbclass +3 -0
+        if os.path.exists(name):
             conf2append = ' %s' % conf
             d.appendVar('FILES:%s' % pkg, conf2append)
             d.appendVar('CONFFILES:%s' % pkg, conf2append)
@@ -121,18 +127,23 @@ python split_kernel_module_packages () {
         # Write out any modconf fragment
         modconflist = (d.getVar("KERNEL_MODULE_PROBECONF") or "").split()
         modconf = d.getVar('module_conf_%s' % basename)
+
+        # The .conf file can either be installed by a recipe or generated

from module_conf_*

meta/classes-recipe/kernel-module-split.bbclass +3 -0
+        conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
+        name = '%s%s' % (dvar, conf)
+        # If module name is in KERNEL_MODULE_PROBECONF, then generate the

.conf file and write to name.
if modconf and basename in modconflist:

meta/classes-recipe/kernel-module-split.bbclass +0 -6
-            conf = '%s/%s.conf' % (d.getVar('modprobedir'), basename)
-            name = '%s%s' % (dvar, conf)
             os.makedirs(os.path.dirname(name), exist_ok=True)
             with open(name, 'w') as f:
                 f.write("%s\n" % modconf)
-            conf2append = ' %s' % conf
-            d.appendVar('FILES:%s' % pkg, conf2append)
-            d.appendVar('CONFFILES:%s' % pkg, conf2append)
-
         elif modconf:
             bb.error("Please ensure module %s is listed in

KERNEL_MODULE_PROBECONF since module_conf_%s is set" % (basename, basename))

meta/classes-recipe/kernel-module-split.bbclass +2 -0
+        # If the .conf file exits, then add it to FILES:* and CONFFILES:*.
+        # It doesn't matter if it was generated from module_conf_* or

installed by the recipe.

meta/classes-recipe/kernel-module-split.bbclass python split_kernel_module_packages () { · line 146

Quoted 4 lines
meta/classes-recipe/kernel-module-split.bbclass +4 -0
+        if os.path.exists(name):
+            conf2append = ' %s' % conf
+            d.appendVar('FILES:%s' % pkg, conf2append)
+            d.appendVar('CONFFILES:%s' % pkg, conf2append)

This version looks good to me.

I would still be nice to have a test for this, but the code paths are sufficiently
similar now that I don't see anything that could break.

Bruce

Quoted 6 lines

if "description" in vals:
old_desc = d.getVar('DESCRIPTION:' + pkg) or ""
--
2.49.0

Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II