kernel.bbclass: fix -dev files when KERNEL_PACKAGE_NAME is PN
PATCH 1/1 kernel.bbclass: fix -dev files when KERNEL_PACKAGE_NAME is PN
With KERNEL_PACKAGE_NAME = "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged and do_package fails:
ERROR: linux-yocto-7.2.8+git-r0 do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:
/boot/System.map-7.2.8-yocto-standard
/boot/config-7.2.8-yocto-standard
/boot/Module.symvers-7.2.8-yocto-standardThis matters when a second kernel recipe. So, Rename the kernel variables to the ${PN} names before the keys are expanded, so the kernel values are kept.
Remove the old warning about bug 11905, which was closed as not a bug.
AI-Generated: Uses Claude
---
Notes:
Tested on master: without this patch do_package fails with the QA error above; with it the build passes with no warnings and linux-yocto-dev contains System.map, Module.symvers and the config.
meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
meta/classes-recipe/kernel.bbclass +14 -7
@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" KERNEL_VERSION_PKG_NAME = "${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}" KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}" +# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg from bitbake.conf +# would replace the kernel values during key expansion, so rename the kernel +# variables to the ${PN} names first. +python kernel_package_name_handler () { + d = e.data + if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"): + return + + for pkg in ("dev", "dbg"): + d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg, "FILES:${PN}-%s" % pkg) +} +addhandler kernel_package_name_handler +kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise" + python __anonymous () { - pn = d.getVar("PN") kpn = d.getVar("KERNEL_PACKAGE_NAME") - # XXX Remove this after bug 11905 is resolved - # FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly - if kpn == pn: - bb.warn("Some packages (E.g. *-dev) might be missing due to " - "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)") - # The default kernel recipe builds in a shared location defined by # bitbake/distro confs: STAGING_KERNEL_DIR and STAGING_KERNEL_BUILDDIR. # Set these variables to directories under ${WORKDIR} in alternate
Signature · 1 line
2.34.1
Quoted 1 line · Prasanth R
Can you elaborate on the configuration / build that you are doing that is triggering this ? You have multiple kernels ? and you are setting KERNEL_PACKAGE_NAME = "${PN}" as part of that ? How you ran into this is important for understanding the context. It could be that we can avoid the issue in another way.
Quoted 4 lines
With
KERNEL_PACKAGE_NAME= "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged anddo_packagefails:
Also for -dbg, the bitbake.conf value is a superset of what the kernel class sets, so nothing is actually lost there. It is just the key collision warning. Worth saying that, so it is clear that -dev is the real breakage.
Quoted 9 lines
ERROR: linux-yocto-7.2.8+git-r0
do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:/boot/System.map-7.2.8-yocto-standard /boot/config-7.2.8-yocto-standard /boot/Module.symvers-7.2.8-yocto-standardThis matters when a second kernel recipe. So, Rename the kernel variables to the ${PN} names before the keys are expanded, so the kernel values are kept.
This sentence is broken. Can you say who actually sets KERNEL_PACKAGE_NAME = "${PN}" and why? I assume an alternate/second kernel recipe, but that should be spelled out. This is the specifics that are hinted at in my opening question.
I'd also like the commit message to explain why an event handler is needed. My first reaction was that this was more complex than it needed to be. But key expansion happens (in sorted order) before anonymous python runs, so by the time __anonymous() sees it, the kernel value is already gone. Fixing it in anonymous python would also clobber any :append to FILES:${KERNEL_PACKAGE_NAME}-dev from a bbappend. A :append in the class keeps the files but still triggers the "Variable key ... replaces original key" warning on every parse. So renaming before expansion is the right place. Just say so in the commit, so the next person doesn't have to work it out again.
Quoted 2 lines
Remove the old warning about bug 11905, which was closed as not a bug.
Fine. 11905 is the same key expansion ordering issue, and that is expected bitbake behaviour, which is why the fix belongs here.
Quoted 9 lines
AI-Generated: Uses Claude
Signed-off-by Prasanth R <[email protected]>---
Notes:
Tested on master: without this patchdo_packagefails with the QA error above; with it the build passes with no warnings and linux-yocto-dev contains System.map, Module.symvers and the config.
Please also confirm the default case (KERNEL_PACKAGE_NAME = "kernel") is unchanged. It should be, since the handler returns early, but it is the case almost everyone uses.
Also note in the commit that with KERNEL_PACKAGE_NAME = "${PN}", the -dev package no longer picks up the bitbake.conf default -dev contents (includedir, .so symlinks, etc). That matches what kernel-dev gets today, so I think it is correct, but it is a behaviour change and should be stated (and confirmed by testing).
Quoted 9 lines
meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)meta/classes-recipe/kernel.bbclass +0
-0
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 65
Quoted 6 lines
meta/classes-recipe/kernel.bbclass +3
-0@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" KERNEL_VERSION_PKG_NAME = "${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}" KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}" +# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg from bitbake.conf +# would replace the kernel values during key expansion, so rename the kernel +# variables to the ${PN} names first.
Same here. Add a line saying this must run before key expansion (i.e. it can't live in __anonymous), so nobody "simplifies" it later.
Quoted 11 lines
meta/classes-recipe/kernel.bbclass +11
-0+python kernel_package_name_handler () { + d = e.data + if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"): + return + + for pkg in ("dev", "dbg"): + d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg, "FILES:${PN}-%s" % pkg) +} +addhandler kernel_package_name_handler +kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise" +
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 72
Quoted 10 lines
meta/classes-recipe/kernel.bbclass +0
-7python __anonymous () { - pn = d.getVar("PN") kpn = d.getVar("KERNEL_PACKAGE_NAME") - # XXX Remove this after bug 11905 is resolved - # FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly - if kpn == pn: - bb.warn("Some packages (E.g. *-dev) might be missing due to " - "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)") -
The code itself looks fine. A v2 with the commit message and comment updates and I'll ack it.
Bruce
Quoted 2 lines
meta/classes-recipe/kernel.bbclass +0
-1--2.34.1
Hi Bruce,
Thanks for the detailed review and the explanation regarding the event handler. That makes perfect sense.
Sent v2.
Thanks,
Prasanth R
Quoted 128 lines · Bruce Ashfield
Quoted 1 line · Prasanth R
Can you elaborate on the configuration / build that you are doing that is triggering this ? You have multiple kernels ? and you are setting
KERNEL_PACKAGE_NAME= "${PN}" as part of that ? How you ran into this is important for understanding the context. It could be that we can avoid the issue in another way.Quoted 4 lines
With
KERNEL_PACKAGE_NAME= "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged anddo_packagefails:Also for -dbg, the bitbake.conf value is a superset of what the kernel class
sets, so nothing is actually lost there. It is just the key collision warning. Worth saying that, so it is clear that -dev is the real breakage.Quoted 9 lines
ERROR: linux-yocto-7.2.8+git-r0
do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:/boot/System.map-7.2.8-yocto-standard /boot/config-7.2.8-yocto-standard /boot/Module.symvers-7.2.8-yocto-standardThis matters when a second kernel recipe. So, Rename the kernel variables to the ${PN} names before the keys are expanded, so the kernel values
are kept.
This sentence is broken. Can you say who actually sets
KERNEL_PACKAGE_NAME= "${PN}" and why? I assume an alternate/second kernel recipe, but that should be spelled out. This is the specifics that are hinted at in my opening question.I'd also like the commit message to explain why an event handler is needed. My first reaction was that this was more complex than it needed to be. But key expansion happens (in sorted order) before anonymous python runs, so by the time
__anonymous()sees it, the kernel value is already gone. Fixing it in anonymous python would also clobber any :append to FILES:${KERNEL_PACKAGE_NAME}-dev from a bbappend. A :append in the class keeps the files but still triggers the "Variable key ... replaces original key" warning on every parse. So renaming before expansion is the right place. Just say so in the commit, so the next person doesn't have to work it out again.Quoted 2 lines
Remove the old warning about bug 11905, which was closed as not a bug.
Fine. 11905 is the same key expansion ordering issue, and that is expected bitbake behaviour, which is why the fix belongs here.
Quoted 8 lines
AI-Generated: Uses Claude
Signed-off-by Prasanth R <[email protected]>---
Notes:
Tested on master: without this patchdo_packagefails with the QA error above; with it the build passes with no warnings andlinux-yocto-dev
Quoted 1 line
contains System.map, Module.symvers and the config.
Please also confirm the default case (
KERNEL_PACKAGE_NAME= "kernel") is unchanged. It should be, since the handler returns early, but it is the case almost everyone uses.Also note in the commit that with
KERNEL_PACKAGE_NAME= "${PN}", the -dev package no longer picks up the bitbake.conf default -dev contents (includedir, .so symlinks, etc). That matches what kernel-dev gets today, so I think it is correct, but it is a behaviour change and should be stated (and confirmed by testing).Review
Quoted 5 lines
meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)(quoted diff) +0
-0b/meta/classes-recipe/kernel.bbclass
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 60
Quoted 5 lines
index
0a6d754108..b018c164fd100644meta/classes-recipe/kernel.bbclass +0
-0@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" KERNEL_VERSION_PKG_NAME ="${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}"
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
Quoted 3 lines
meta/classes-recipe/kernel.bbclass +1
-0KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}" +# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg frombitbake.conf
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 64
Quoted 1 line
meta/classes-recipe/kernel.bbclass +1
-0+# would replace the kernel values during key expansion, so rename thekernel
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 65
Quoted 1 line
meta/classes-recipe/kernel.bbclass +1
-0+# variables to the ${PN} names first.Same here. Add a line saying this must run before key expansion (i.e. it can't live in
__anonymous), so nobody "simplifies" it later.meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
Quoted 7 lines
meta/classes-recipe/kernel.bbclass +7
-0+python kernel_package_name_handler () { + d = e.data + if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"): + return + + for pkg in ("dev", "dbg"): + d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg,"FILES:${PN}-%s" % pkg)
Quoted 4 lines
meta/classes-recipe/kernel.bbclass +4
-0+} +addhandler kernel_package_name_handler +kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise" +meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 72
Quoted 10 lines
meta/classes-recipe/kernel.bbclass +0
-7python __anonymous () { - pn = d.getVar("PN") kpn = d.getVar("KERNEL_PACKAGE_NAME") - # XXX Remove this after bug 11905 is resolved - # FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly - if kpn == pn: - bb.warn("Some packages (E.g. *-dev) might be missing due to " - "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)") -The code itself looks fine. A v2 with the commit message and comment updates and I'll ack it.
Bruce
Quoted 2 lines
meta/classes-recipe/kernel.bbclass +0
-1--2.34.1
Hi Bruce,
I don't see my patches in the patchwork, am i missing something?
Thanks,
Prasanth R
Quoted 145 lines · Prasanth Ramesh
Hi Bruce,
Thanks for the detailed review and the explanation regarding the event handler. That makes perfect sense.
Sent v2.
Thanks,
Prasanth RQuoted 130 lines · Bruce Ashfield
Quoted 1 line · Prasanth R
Can you elaborate on the configuration / build that you are doing that is triggering this ? You have multiple kernels ? and you are setting
KERNEL_PACKAGE_NAME= "${PN}" as part of that ? How you ran into this is important for understanding the context. It could be that we can avoid the issue in another way.Quoted 4 lines
With
KERNEL_PACKAGE_NAME= "${PN}", FILES:${KERNEL_PACKAGE_NAME}-dev and FILES:${PN}-dev from bitbake.conf become the same variable and the bitbake.conf value wins. The same happens for -dbg. The kernel files are then not packaged anddo_packagefails:Also for -dbg, the bitbake.conf value is a superset of what the kernel class
sets, so nothing is actually lost there. It is just the key collision warning. Worth saying that, so it is clear that -dev is the real breakage.Quoted 8 lines
ERROR: linux-yocto-7.2.8+git-r0
do_package: QA Issue: linux-yocto:
Files/directories were installed but not shipped in any package:/boot/System.map-7.2.8-yocto-standard /boot/config-7.2.8-yocto-standard /boot/Module.symvers-7.2.8-yocto-standardThis matters when a second kernel recipe. So, Rename the kernel
variables
Quoted 1 line
to the ${PN} names before the keys are expanded, so the kernel values
are kept.
This sentence is broken. Can you say who actually sets
KERNEL_PACKAGE_NAME= "${PN}" and why? I assume an alternate/second kernel recipe, but that should be spelled out. This is the specifics that are hinted at in my opening question.I'd also like the commit message to explain why an event handler is needed. My first reaction was that this was more complex than it needed to be. But key expansion happens (in sorted order) before anonymous python runs, so by the time
__anonymous()sees it, the kernel value is already gone. Fixing it in anonymous python would also clobber any :append to FILES:${KERNEL_PACKAGE_NAME}-dev from a bbappend. A :append in the class keeps the files but still triggers the "Variable key ... replaces original key" warning on every parse. So renaming before expansion is the right place. Just say so in the commit, so the next person doesn't have to work it out again.Quoted 2 lines
Remove the old warning about bug 11905, which was closed as not a bug.
Fine. 11905 is the same key expansion ordering issue, and that is expected bitbake behaviour, which is why the fix belongs here.
Quoted 8 lines
AI-Generated: Uses Claude
Signed-off-by Prasanth R <[email protected]>---
Notes:
Tested on master: without this patchdo_packagefails with the QA error above; with it the build passes with no warnings andlinux-yocto-dev
Quoted 1 line
contains System.map, Module.symvers and the config.
Please also confirm the default case (
KERNEL_PACKAGE_NAME= "kernel") is unchanged. It should be, since the handler returns early, but it is the case almost everyone uses.Also note in the commit that with
KERNEL_PACKAGE_NAME= "${PN}", the -dev package no longer picks up the bitbake.conf default -dev contents (includedir, .so symlinks, etc). That matches what kernel-dev gets today, so I think it is correct, but it is a behaviour change and should be stated (and confirmed by testing).Review
Quoted 5 lines
meta/classes-recipe/kernel.bbclass | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)(quoted diff) +0
-0b/meta/classes-recipe/kernel.bbclass
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] =
Quoted 4 lines
index
0a6d754108..b018c164fd100644meta/classes-recipe/kernel.bbclass +0
-0@@ -60,16 +60,23 @@ KERNEL_VERSION_NAME[vardepvalue] ="${LINUX_VERSION}"
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = · line 60
Quoted 1 line
meta/classes-recipe/kernel.bbclass +0
-0KERNEL_VERSION_PKG_NAME ="${@legitimize_package_name(d.getVar('KERNEL_VERSION'))}"
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
Quoted 3 lines
meta/classes-recipe/kernel.bbclass +1
-0KERNEL_VERSION_PKG_NAME[vardepvalue] = "${LINUX_VERSION}" +# With KERNEL_PACKAGE_NAME = "${PN}", FILES:${PN}-dev/-dbg frombitbake.conf
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = · line 64
Quoted 1 line
meta/classes-recipe/kernel.bbclass +1
-0+# would replace the kernel values during key expansion, so rename thekernel
meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 65
Quoted 1 line
meta/classes-recipe/kernel.bbclass +1
-0+# variables to the ${PN} names first.Same here. Add a line saying this must run before key expansion (i.e. it can't live in
__anonymous), so nobody "simplifies" it later.meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}"
Quoted 7 lines
meta/classes-recipe/kernel.bbclass +7
-0+python kernel_package_name_handler () { + d = e.data + if d.getVar("KERNEL_PACKAGE_NAME") != d.getVar("PN"): + return + + for pkg in ("dev", "dbg"): + d.renameVar("FILES:${KERNEL_PACKAGE_NAME}-%s" % pkg,"FILES:${PN}-%s" % pkg)
Quoted 4 lines
meta/classes-recipe/kernel.bbclass +4
-0+} +addhandler kernel_package_name_handler +kernel_package_name_handler[eventmask] = "bb.event.RecipePreFinalise" +meta/classes-recipe/kernel.bbclass KERNEL_VERSION_NAME[vardepvalue] = "${LINUX_VERSION}" · line 72
Quoted 10 lines
meta/classes-recipe/kernel.bbclass +0
-7python __anonymous () { - pn = d.getVar("PN") kpn = d.getVar("KERNEL_PACKAGE_NAME") - # XXX Remove this after bug 11905 is resolved - # FILES:${KERNEL_PACKAGE_NAME}-dev doesn't expand correctly - if kpn == pn: - bb.warn("Some packages (E.g. *-dev) might be missing due to " - "bug 11905 (variable KERNEL_PACKAGE_NAME == PN)") -The code itself looks fine. A v2 with the commit message and comment updates and I'll ack it.
Bruce
Quoted 2 lines
meta/classes-recipe/kernel.bbclass +0
-1--2.34.1