Skip to contentSkip to conversation

kernel.bbclass: fix -dev files when KERNEL_PACKAGE_NAME is PN

PATCH In review prasanth 2 participants 4 messages Oct 29

lore
Share & tools
lore

prasanth

author
#0 lore raw

PATCH 1/1 kernel.bbclass: fix -dev files when KERNEL_PACKAGE_NAME is PN

With KERNEL_PACKAGE_NAME = "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged and do_package fails:

ERROR: linux-yocto-7.2.8+git-r0 do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:

    /boot/System.map-7.2.8-yocto-standard
    /boot/config-7.2.8-yocto-standard
    /boot/Module.symvers-7.2.8-yocto-standard

This matters when a second kernel recipe. So, Rename the kernel variables to the ${PN} names before the keys are expanded, so the kernel values are kept.

Remove the old warning about bug 11905, which was closed as not a bug.

AI-Generated: Uses Claude

---

Notes:
Tested on master: without this patch do_package fails with the QA error above; with it the build passes with no warnings and linux-yocto-dev contains System.map, Module.symvers and the config.

 meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)
meta/classes-recipe/kernel.bbclass +14 -7
index 0a6d754108..b018c164fd 100644
@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
 KERNEL_VERSION_PKG_NAME = "${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}"
 KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}"
 
+# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg from bitbake.conf
+# would replace the kernel values during key expansion, so rename the kernel
+# variables to the ${PN} names first.
+python kernel_package_name_handler () {
+    d = e.data
+    if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"):
+        return
+
+    for pkg in ("dev", "dbg"):
+        d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg, "FILES:${PN}-%s" % pkg)
+}
+addhandler kernel_package_name_handler
+kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise"
+
 python __anonymous () {
-    pn = d.getVar("PN")
     kpn = d.getVar("KERNEL_PACKAGE_NAME")
 
-    # XXX Remove this after bug 11905 is resolved
-    #  FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly
-    if kpn == pn:
-        bb.warn("Some packages (E.g. *-dev) might be missing due to "
-                "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)")
-
     # The default kernel recipe builds in a shared location defined by
     # bitbake/distro confs: STAGING_KERNEL_DIR and STAGING_KERNEL_BUILDDIR.
     # Set these variables to directories under ${WORKDIR} in alternate
Signature · 1 line
2.34.1
#1 lore raw
Quoted 1 line · Prasanth R

Can you elaborate on the configuration / build that you are doing that is triggering this ? You have multiple kernels ? and you are setting KERNEL_PACKAGE_NAME = "${PN}" as part of that ? How you ran into this is important for understanding the context. It could be that we can avoid the issue in another way.

Quoted 4 lines

With KERNEL_PACKAGE_NAME = "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged and do_package fails:

Also for -dbg, the bitbake.conf value is a superset of what the kernel class sets, so nothing is actually lost there. It is just the key collision warning. Worth saying that, so it is clear that -dev is the real breakage.

Quoted 9 lines

ERROR: linux-yocto-7.2.8+git-r0 do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:

    /boot/System.map-7.2.8-yocto-standard
    /boot/config-7.2.8-yocto-standard
    /boot/Module.symvers-7.2.8-yocto-standard

This matters when a second kernel recipe. So, Rename the kernel variables to the ${PN} names before the keys are expanded, so the kernel values are kept.

This sentence is broken. Can you say who actually sets KERNEL_PACKAGE_NAME = "${PN}" and why? I assume an alternate/second kernel recipe, but that should be spelled out. This is the specifics that are hinted at in my opening question.

I'd also like the commit message to explain why an event handler is needed. My first reaction was that this was more complex than it needed to be. But key expansion happens (in sorted order) before anonymous python runs, so by the time __anonymous() sees it, the kernel value is already gone. Fixing it in anonymous python would also clobber any :append to FILES:${KERNEL_PACKAGE_NAME}-dev from a bbappend. A :append in the class keeps the files but still triggers the "Variable key ... replaces original key" warning on every parse. So renaming before expansion is the right place. Just say so in the commit, so the next person doesn't have to work it out again.

Quoted 2 lines

Remove the old warning about bug 11905, which was closed as not a bug.

Fine. 11905 is the same key expansion ordering issue, and that is expected bitbake behaviour, which is why the fix belongs here.

Quoted 9 lines

AI-Generated: Uses Claude

---

Notes:
Tested on master: without this patch do_package fails with the QA error above; with it the build passes with no warnings and linux-yocto-dev contains System.map, Module.symvers and the config.

Please also confirm the default case (KERNEL_PACKAGE_NAME = "kernel") is unchanged. It should be, since the handler returns early, but it is the case almost everyone uses.

Also note in the commit that with KERNEL_PACKAGE_NAME = "${PN}", the -dev package no longer picks up the bitbake.conf default -dev contents (includedir, .so symlinks, etc). That matches what kernel-dev gets today, so I think it is correct, but it is a behaviour change and should be stated (and confirmed by testing).

Quoted 9 lines
 meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)
meta/classes-recipe/kernel.bbclass +0 -0
index 0a6d754108..b018c164fd 100644

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 65

Quoted 6 lines
meta/classes-recipe/kernel.bbclass +3 -0
@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
 KERNEL_VERSION_PKG_NAME = "${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}"
 KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}"
 
+# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg from bitbake.conf
+# would replace the kernel values during key expansion, so rename the kernel
+# variables to the ${PN} names first.

Same here. Add a line saying this must run before key expansion (i.e. it can't live in __anonymous), so nobody "simplifies" it later.

Quoted 11 lines
meta/classes-recipe/kernel.bbclass +11 -0
+python kernel_package_name_handler () {
+    d = e.data
+    if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"):
+        return
+
+    for pkg in ("dev", "dbg"):
+        d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg, "FILES:${PN}-%s" % pkg)
+}
+addhandler kernel_package_name_handler
+kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise"
+

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 72

Quoted 10 lines
meta/classes-recipe/kernel.bbclass +0 -7
 python __anonymous () {
-    pn = d.getVar("PN")
     kpn = d.getVar("KERNEL_PACKAGE_NAME")
 
-    # XXX Remove this after bug 11905 is resolved
-    #  FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly
-    if kpn == pn:
-        bb.warn("Some packages (E.g. *-dev) might be missing due to "
-                "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)")
-

The code itself looks fine. A v2 with the commit message and comment updates and I'll ack it.

Bruce

Quoted 2 lines
meta/classes-recipe/kernel.bbclass +0 -1
--

2.34.1

author
#2 lore raw

Hi Bruce,

Thanks for the detailed review and the explanation regarding the event handler. That makes perfect sense.

Sent v2.

Thanks,
Prasanth R

Quoted 128 lines · Bruce Ashfield
Quoted 1 line · Prasanth R

Can you elaborate on the configuration / build that you are doing that is triggering this ? You have multiple kernels ? and you are setting KERNEL_PACKAGE_NAME = "${PN}" as part of that ? How you ran into this is important for understanding the context. It could be that we can avoid the issue in another way.

Quoted 4 lines

With KERNEL_PACKAGE_NAME = "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged and do_package fails:

Also for -dbg, the bitbake.conf value is a superset of what the kernel class
sets, so nothing is actually lost there. It is just the key collision warning. Worth saying that, so it is clear that -dev is the real breakage.

Quoted 9 lines

ERROR: linux-yocto-7.2.8+git-r0 do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:

    /boot/System.map-7.2.8-yocto-standard
    /boot/config-7.2.8-yocto-standard
    /boot/Module.symvers-7.2.8-yocto-standard

This matters when a second kernel recipe. So, Rename the kernel variables to the ${PN} names before the keys are expanded, so the kernel values

are kept.

This sentence is broken. Can you say who actually sets KERNEL_PACKAGE_NAME = "${PN}" and why? I assume an alternate/second kernel recipe, but that should be spelled out. This is the specifics that are hinted at in my opening question.

I'd also like the commit message to explain why an event handler is needed. My first reaction was that this was more complex than it needed to be. But key expansion happens (in sorted order) before anonymous python runs, so by the time __anonymous() sees it, the kernel value is already gone. Fixing it in anonymous python would also clobber any :append to FILES:${KERNEL_PACKAGE_NAME}-dev from a bbappend. A :append in the class keeps the files but still triggers the "Variable key ... replaces original key" warning on every parse. So renaming before expansion is the right place. Just say so in the commit, so the next person doesn't have to work it out again.

Quoted 2 lines

Remove the old warning about bug 11905, which was closed as not a bug.

Fine. 11905 is the same key expansion ordering issue, and that is expected bitbake behaviour, which is why the fix belongs here.

Quoted 8 lines

AI-Generated: Uses Claude

---

Notes:
Tested on master: without this patch do_package fails with the QA error above; with it the build passes with no warnings and

linux-yocto-dev

Quoted 1 line

contains System.map, Module.symvers and the config.

Please also confirm the default case (KERNEL_PACKAGE_NAME = "kernel") is unchanged. It should be, since the handler returns early, but it is the case almost everyone uses.

Also note in the commit that with KERNEL_PACKAGE_NAME = "${PN}", the -dev package no longer picks up the bitbake.conf default -dev contents (includedir, .so symlinks, etc). That matches what kernel-dev gets today, so I think it is correct, but it is a behaviour change and should be stated (and confirmed by testing).

Review

Quoted 5 lines
 meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)
(quoted diff) +0 -0

b/meta/classes-recipe/kernel.bbclass

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 60

Quoted 5 lines

index 0a6d754108..b018c164fd 100644

meta/classes-recipe/kernel.bbclass +0 -0
@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
 KERNEL_VERSION_PKG_NAME =

"${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}"

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"

Quoted 3 lines
meta/classes-recipe/kernel.bbclass +1 -0
 KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}"
 
+# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg from

bitbake.conf

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 64

Quoted 1 line
meta/classes-recipe/kernel.bbclass +1 -0
+# would replace the kernel values during key expansion, so rename the

kernel

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 65

Quoted 1 line
meta/classes-recipe/kernel.bbclass +1 -0
+# variables to the ${PN} names first.

Same here. Add a line saying this must run before key expansion (i.e. it can't live in __anonymous), so nobody "simplifies" it later.

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"

Quoted 7 lines
meta/classes-recipe/kernel.bbclass +7 -0
+python kernel_package_name_handler () {
+    d = e.data
+    if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"):
+        return
+
+    for pkg in ("dev", "dbg"):
+        d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg,

"FILES:${PN}-%s" % pkg)

Quoted 4 lines
meta/classes-recipe/kernel.bbclass +4 -0
+}
+addhandler kernel_package_name_handler
+kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise"
+

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 72

Quoted 10 lines
meta/classes-recipe/kernel.bbclass +0 -7
 python __anonymous () {
-    pn = d.getVar("PN")
     kpn = d.getVar("KERNEL_PACKAGE_NAME")
 
-    # XXX Remove this after bug 11905 is resolved
-    #  FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly
-    if kpn == pn:
-        bb.warn("Some packages (E.g. *-dev) might be missing due to "
-                "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)")
-

The code itself looks fine. A v2 with the commit message and comment updates and I'll ack it.

Bruce

Quoted 2 lines
meta/classes-recipe/kernel.bbclass +0 -1
--

2.34.1

#3 lore raw

Hi Bruce,

I don't see my patches in the patchwork, am i missing something?

Thanks,
Prasanth R

Quoted 145 lines · Prasanth Ramesh

Hi Bruce,

Thanks for the detailed review and the explanation regarding the event handler. That makes perfect sense.

Sent v2.

Thanks,
Prasanth R

Quoted 130 lines · Bruce Ashfield
Quoted 1 line · Prasanth R

Can you elaborate on the configuration / build that you are doing that is triggering this ? You have multiple kernels ? and you are setting KERNEL_PACKAGE_NAME = "${PN}" as part of that ? How you ran into this is important for understanding the context. It could be that we can avoid the issue in another way.

Quoted 4 lines

With KERNEL_PACKAGE_NAME = "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged and do_package fails:

Also for -dbg, the bitbake.conf value is a superset of what the kernel class
sets, so nothing is actually lost there. It is just the key collision warning. Worth saying that, so it is clear that -dev is the real breakage.

Quoted 8 lines

ERROR: linux-yocto-7.2.8+git-r0 do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:

    /boot/System.map-7.2.8-yocto-standard
    /boot/config-7.2.8-yocto-standard
    /boot/Module.symvers-7.2.8-yocto-standard

This matters when a second kernel recipe. So, Rename the kernel

variables

Quoted 1 line

to the ${PN} names before the keys are expanded, so the kernel values

are kept.

This sentence is broken. Can you say who actually sets KERNEL_PACKAGE_NAME = "${PN}" and why? I assume an alternate/second kernel recipe, but that should be spelled out. This is the specifics that are hinted at in my opening question.

I'd also like the commit message to explain why an event handler is needed. My first reaction was that this was more complex than it needed to be. But key expansion happens (in sorted order) before anonymous python runs, so by the time __anonymous() sees it, the kernel value is already gone. Fixing it in anonymous python would also clobber any :append to FILES:${KERNEL_PACKAGE_NAME}-dev from a bbappend. A :append in the class keeps the files but still triggers the "Variable key ... replaces original key" warning on every parse. So renaming before expansion is the right place. Just say so in the commit, so the next person doesn't have to work it out again.

Quoted 2 lines

Remove the old warning about bug 11905, which was closed as not a bug.

Fine. 11905 is the same key expansion ordering issue, and that is expected bitbake behaviour, which is why the fix belongs here.

Quoted 8 lines

AI-Generated: Uses Claude

---

Notes:
Tested on master: without this patch do_package fails with the QA error above; with it the build passes with no warnings and

linux-yocto-dev

Quoted 1 line

contains System.map, Module.symvers and the config.

Please also confirm the default case (KERNEL_PACKAGE_NAME = "kernel") is unchanged. It should be, since the handler returns early, but it is the case almost everyone uses.

Also note in the commit that with KERNEL_PACKAGE_NAME = "${PN}", the -dev package no longer picks up the bitbake.conf default -dev contents (includedir, .so symlinks, etc). That matches what kernel-dev gets today, so I think it is correct, but it is a behaviour change and should be stated (and confirmed by testing).

Review

Quoted 5 lines
 meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)
(quoted diff) +0 -0

b/meta/classes-recipe/kernel.bbclass

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] =

Quoted 4 lines

index 0a6d754108..b018c164fd 100644

meta/classes-recipe/kernel.bbclass +0 -0
@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] =

"${LINUX_VERSION}"

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = · line 60

Quoted 1 line
meta/classes-recipe/kernel.bbclass +0 -0
 KERNEL_VERSION_PKG_NAME =

"${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}"

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"

Quoted 3 lines
meta/classes-recipe/kernel.bbclass +1 -0
 KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}"
 
+# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg from

bitbake.conf

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = · line 64

Quoted 1 line
meta/classes-recipe/kernel.bbclass +1 -0
+# would replace the kernel values during key expansion, so rename the

kernel

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 65

Quoted 1 line
meta/classes-recipe/kernel.bbclass +1 -0
+# variables to the ${PN} names first.

Same here. Add a line saying this must run before key expansion (i.e. it can't live in __anonymous), so nobody "simplifies" it later.

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"

Quoted 7 lines
meta/classes-recipe/kernel.bbclass +7 -0
+python kernel_package_name_handler () {
+    d = e.data
+    if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"):
+        return
+
+    for pkg in ("dev", "dbg"):
+        d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg,

"FILES:${PN}-%s" % pkg)

Quoted 4 lines
meta/classes-recipe/kernel.bbclass +4 -0
+}
+addhandler kernel_package_name_handler
+kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise"
+

meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 72

Quoted 10 lines
meta/classes-recipe/kernel.bbclass +0 -7
 python __anonymous () {
-    pn = d.getVar("PN")
     kpn = d.getVar("KERNEL_PACKAGE_NAME")
 
-    # XXX Remove this after bug 11905 is resolved
-    #  FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly
-    if kpn == pn:
-        bb.warn("Some packages (E.g. *-dev) might be missing due to "
-                "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)")
-

The code itself looks fine. A v2 with the commit message and comment updates and I'll ack it.

Bruce

Quoted 2 lines
meta/classes-recipe/kernel.bbclass +0 -1
--

2.34.1