kernel.bbclass: copy some more needed files in do_shared_workdir
PATCH 1/1 kernel.bbclass: copy some more needed files in do_shared_workdir
We need to copy include/config/auto.conf into ${STAGING_KERNEL_BUILDDIR} to avoid module compilation errors like this:
| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source'
| make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts'
|
| ERROR: Kernel configuration is invalid.
| include/generated/autoconf.h or include/config/auto.conf are missing.
| Run 'make oldconfig && make prepare' on kernel src to fix it.
|
| make[2]: *** [build/tmp-glibc/work-shared/machine/kernel-source/Makefile:641: include/config/auto.conf] Error 1We also need scripts/basic/fixdep to avoid errors like this:
| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source'
| make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts'
| CC [M] build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o
| /bin/sh: scripts/basic/fixdep: No such file or directory
| make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.build:303: build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o] Error 127And scripts/mod/modpost to avoid errors like this:
| Building modules, stage 2.
| MODPOST 1 modules
| /bin/sh: scripts/mod/modpost: No such file or directory
| make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.modpost:92: __modpost] Error 127Most of the time, the absence of these files is masked by the make-mod-scripts recipe's do_configure task, which causes a whole bunch of extra files to be built into ${STAGING_KERNEL_BUILDDIR} (including these two). Typically make-mod-scripts will be built before any kernel modules, and so it didn't matter that do_shared_workdir was missing these files.
However the problem will be evident if you do certain sequences of tasks, for example a complete build first (so that make-mod-scripts is already built), then:
bitbake linux-yocto my-kernel-module -c clean
bitbake my-kernel-module -c install
--- meta/classes/kernel.bbclass | 5 +++++ 1 file changed, 5 insertions(+)
meta/classes/kernel.bbclass +5 -0
@@ -452,6 +452,7 @@ do_shared_workdir () { cp .config $kerneldir/ mkdir -p $kerneldir/include/config cp include/config/kernel.release $kerneldir/include/config/kernel.release + cp include/config/auto.conf $kerneldir/include/config/ if [ -e certs/signing_key.x509 ]; then # The signing_key.* files are stored in the certs/ dir in # newer Linux kernels
@@ -460,6 +461,10 @@ do_shared_workdir () { elif [ -e signing_key.priv ]; then cp signing_key.* $kerneldir/ fi + mkdir -p $kerneldir/scripts/basic + cp scripts/basic/fixdep $kerneldir/scripts/basic/ + mkdir -p $kerneldir/scripts/mod + cp scripts/mod/modpost $kerneldir/scripts/mod/ # We can also copy over all the generated files and avoid special cases # like version.h, but we've opted to keep this small until file creep starts
Signature · 1 line
2.21.1
Quoted 28 lines · Dan Callaghan
We need to copy
include/config/auto.confinto ${STAGING_KERNEL_BUILDDIR} to avoid module compilation errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | | ERROR: Kernel configuration is invalid. | include/generated/autoconf.h or include/config/auto.conf are missing. | Run 'make oldconfig && make prepare' on kernel src to fix it. | | make[2]: *** [build/tmp-glibc/work-shared/machine/kernel-source/Makefile:641: include/config/auto.conf] Error 1We also need
scripts/basic/fixdepto avoid errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | CC [M] build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o | /bin/sh: scripts/basic/fixdep: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.build:303: build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o] Error 127And
scripts/mod/modpostto avoid errors like this:| Building modules, stage 2. | MODPOST 1 modules | /bin/sh: scripts/mod/modpost: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.modpost:92: __modpost] Error 127
Quoted 7 lines
Most of the time, the absence of these files is masked by the make-mod-scripts recipe's
do_configuretask, which causes a whole bunch of extra files to be built into ${STAGING_KERNEL_BUILDDIR} (including these two). Typically make-mod-scripts will be built before any kernel modules, and so it didn't matter thatdo_shared_workdirwas missing these files.
The design is that, make-mod-scripts is what provides those files, we don't want to copy them, since it is an ever incomplete list (more so in the past, but the concept still holds).
Quoted 6 lines
However the problem will be evident if you do certain sequences of tasks, for example a complete build first (so that make-mod-scripts is already built), then:
bitbake linux-yocto my-kernel-module -c clean
bitbake my-kernel-module -c install
Why can't we fix the dependency ? We chased a few of these around when the mod-scripts recipe was introduced, and this just looks like a variant we missed.
Bruce
Quoted 33 lines
Signed-off-by Dan Callaghan <[email protected]>--- meta/classes/kernel.bbclass | 5 +++++ 1 file changed, 5 insertions(+)meta/classes/kernel.bbclass +5
-1@@ -452,6 +452,7 @@ do_shared_workdir () { cp .config $kerneldir/ mkdir -p $kerneldir/include/config cp include/config/kernel.release $kerneldir/include/config/kernel.release + cp include/config/auto.conf $kerneldir/include/config/ if [ -e certs/signing_key.x509 ]; then # The signing_key.* files are stored in the certs/ dir in # newer Linux kernels@@ -460,6 +461,10 @@ do_shared_workdir () { elif [ -e signing_key.priv ]; then cp signing_key.* $kerneldir/ fi + mkdir -p $kerneldir/scripts/basic + cp scripts/basic/fixdep $kerneldir/scripts/basic/ + mkdir -p $kerneldir/scripts/mod + cp scripts/mod/modpost $kerneldir/scripts/mod/ # We can also copy over all the generated files and avoid special cases # like version.h, but we've opted to keep this small until file creep starts --2.21.1
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Excerpts from Bruce Ashfield's message of 2020-04-21 08:33:34 -04:00:
Quoted 38 lines
Quoted 28 lines · Dan Callaghan
We need to copy
include/config/auto.confinto ${STAGING_KERNEL_BUILDDIR} to avoid module compilation errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | | ERROR: Kernel configuration is invalid. | include/generated/autoconf.h or include/config/auto.conf are missing. | Run 'make oldconfig && make prepare' on kernel src to fix it. | | make[2]: *** [build/tmp-glibc/work-shared/machine/kernel-source/Makefile:641: include/config/auto.conf] Error 1We also need
scripts/basic/fixdepto avoid errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | CC [M] build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o | /bin/sh: scripts/basic/fixdep: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.build:303: build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o] Error 127And
scripts/mod/modpostto avoid errors like this:| Building modules, stage 2. | MODPOST 1 modules | /bin/sh: scripts/mod/modpost: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.modpost:92: __modpost] Error 127Quoted 7 lines
Most of the time, the absence of these files is masked by the make-mod-scripts recipe's
do_configuretask, which causes a whole bunch of extra files to be built into ${STAGING_KERNEL_BUILDDIR} (including these two). Typically make-mod-scripts will be built before any kernel modules, and so it didn't matter thatdo_shared_workdirwas missing these files.
Quoted 3 lines
The design is that, make-mod-scripts is what provides those files, we don't want to copy them, since it is an ever incomplete list (more so in the past, but the concept still holds).
Okay, I think you're saying that what do_shared_workdir claims to do (populate ${STAGING_KERNEL_BUILDDIR} with whatever files are needed to build kernel modules) should actually be handled by make-mod-scripts instead.
Does that mean, do_shared_workdir has no purpose? Or is it doing something else I don't realise?
Quoted 10 lines
Quoted 6 lines
However the problem will be evident if you do certain sequences of tasks, for example a complete build first (so that make-mod-scripts is already built), then:
bitbake linux-yocto my-kernel-module -c clean
bitbake my-kernel-module -c installWhy can't we fix the dependency ? We chased a few of these around when the mod-scripts recipe was introduced, and this just looks like a variant we missed.
Right, so there *is* already a correct dependency on make-mod-scripts:do_compile. The problem is, in the scenario I gave above, that task is already done and Bitbake is not going to run it again. But in fact, the stuff it left behind in ${STAGING_KERNEL_BUILDDIR} was cleaned out when I cleaned the linux-yocto recipe.
I guess the real issue is that we have this "shared" work directory which a bunch of different recipes contribute files to, but can be cleaned out by running the "clean" task on any one of several different recipes.
Signature · 3 lines
Dan Callaghan <[email protected]> Software Engineer Opengear <https://opengear.com/>
Quoted 44 lines · Dan Callaghan
Excerpts from Bruce Ashfield's message of 2020-04-21 08:33:34 -04:00:
Quoted 38 lines
Quoted 28 lines · Dan Callaghan
We need to copy
include/config/auto.confinto ${STAGING_KERNEL_BUILDDIR} to avoid module compilation errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | | ERROR: Kernel configuration is invalid. | include/generated/autoconf.h or include/config/auto.conf are missing. | Run 'make oldconfig && make prepare' on kernel src to fix it. | | make[2]: *** [build/tmp-glibc/work-shared/machine/kernel-source/Makefile:641: include/config/auto.conf] Error 1We also need
scripts/basic/fixdepto avoid errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | CC [M] build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o | /bin/sh: scripts/basic/fixdep: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.build:303: build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o] Error 127And
scripts/mod/modpostto avoid errors like this:| Building modules, stage 2. | MODPOST 1 modules | /bin/sh: scripts/mod/modpost: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.modpost:92: __modpost] Error 127Quoted 7 lines
Most of the time, the absence of these files is masked by the make-mod-scripts recipe's
do_configuretask, which causes a whole bunch of extra files to be built into ${STAGING_KERNEL_BUILDDIR} (including these two). Typically make-mod-scripts will be built before any kernel modules, and so it didn't matter thatdo_shared_workdirwas missing these files.Quoted 3 lines
The design is that, make-mod-scripts is what provides those files, we don't want to copy them, since it is an ever incomplete list (more so in the past, but the concept still holds).
Quoted 7 lines
Okay, I think you're saying that what
do_shared_workdirclaims to do (populate ${STAGING_KERNEL_BUILDDIR} with whatever files are needed to build kernel modules) should actually be handled by make-mod-scripts instead.Does that mean,
do_shared_workdirhas no purpose? Or is it doing something else I don't realise?
How would what populates it change the purpose of what it does ? Just like our SDK and on target module support, those files are generated by the kernel build itself, not copied.
shared-workdir was created to stop slinging around the sheer number of files in a kernel build, it was causing us severe build performance issues.
Having the smallest possible list of explicitly copied files is the goal. The lists are fragile, and inevitably need constant tweaking. Witness the messing around with kernel-devsrc that I have to do, and the only reason that is tolerated is to make the minimal possible development components.
Quoted 12 lines
Quoted 11 lines
Quoted 7 lines
However the problem will be evident if you do certain sequences of tasks, for example a complete build first (so that make-mod-scripts is already built), then:
bitbake linux-yocto my-kernel-module -c clean
bitbake my-kernel-module -c installWhy can't we fix the dependency ? We chased a few of these around when the mod-scripts recipe was introduced, and this just looks like a variant we missed.
Quoted 6 lines
Right, so there *is* already a correct dependency on make-mod-scripts:do_compile. The problem is, in the scenario I gave above, that task is already done and Bitbake is not going to run it again. But in fact, the stuff it left behind in ${STAGING_KERNEL_BUILDDIR} was cleaned out when I cleaned the linux-yocto recipe.
Yes, and that clean is supposed to invalidate things, and trigger the re-run.
Quoted 5 lines
I guess the real issue is that we have this "shared" work directory which a bunch of different recipes contribute files to, but can be cleaned out by running the "clean" task on any one of several different recipes.
We've solved that issue in the past, I'll have a look on Wednesday to see what may have changed and (re)broken it. There may be a bugzilla floating around for that, I'll see if I can track it down as well.
Bruce
Quoted 5 lines
--
Dan Callaghan <[email protected]>
Software Engineer
Opengear <https://opengear.com/>
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 87 lines · Bruce Ashfield via lists.openembedded.org
Quoted 44 lines · Dan Callaghan
Excerpts from Bruce Ashfield's message of 2020-04-21 08:33:34 -04:00:
Quoted 38 lines
Quoted 28 lines · Dan Callaghan
We need to copy
include/config/auto.confinto ${STAGING_KERNEL_BUILDDIR} to avoid module compilation errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | | ERROR: Kernel configuration is invalid. | include/generated/autoconf.h or include/config/auto.conf are missing. | Run 'make oldconfig && make prepare' on kernel src to fix it. | | make[2]: *** [build/tmp-glibc/work-shared/machine/kernel-source/Makefile:641: include/config/auto.conf] Error 1We also need
scripts/basic/fixdepto avoid errors like this:| make[1]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-source' | make[2]: Entering directory 'build/tmp-glibc/work-shared/machine/kernel-build-artifacts' | CC [M] build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o | /bin/sh: scripts/basic/fixdep: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.build:303: build/tmp-glibc/work/vendor-linux/tehuti-module/0.3.6-r0/git/TLK10232_phy_Linux.o] Error 127And
scripts/mod/modpostto avoid errors like this:| Building modules, stage 2. | MODPOST 1 modules | /bin/sh: scripts/mod/modpost: No such file or directory | make[3]: *** [build/tmp-glibc/work-shared/machine/kernel-source/scripts/Makefile.modpost:92: __modpost] Error 127Quoted 7 lines
Most of the time, the absence of these files is masked by the make-mod-scripts recipe's
do_configuretask, which causes a whole bunch of extra files to be built into ${STAGING_KERNEL_BUILDDIR} (including these two). Typically make-mod-scripts will be built before any kernel modules, and so it didn't matter thatdo_shared_workdirwas missing these files.Quoted 3 lines
The design is that, make-mod-scripts is what provides those files, we don't want to copy them, since it is an ever incomplete list (more so in the past, but the concept still holds).
Quoted 7 lines
Okay, I think you're saying that what
do_shared_workdirclaims to do (populate ${STAGING_KERNEL_BUILDDIR} with whatever files are needed to build kernel modules) should actually be handled by make-mod-scripts instead.Does that mean,
do_shared_workdirhas no purpose? Or is it doing something else I don't realise?How would what populates it change the purpose of what it does ? Just like our SDK and on target module support, those files are generated by the kernel build itself, not copied.
shared-workdir was created to stop slinging around the sheer number of files in a kernel build, it was causing us severe build performance issues.
Having the smallest possible list of explicitly copied files is the goal. The lists are fragile, and inevitably need constant tweaking. Witness the messing around with kernel-devsrc that I have to do, and the only reason that is tolerated is to make the minimal possible development components.
Quoted 12 lines
Quoted 11 lines
Quoted 7 lines
However the problem will be evident if you do certain sequences of tasks, for example a complete build first (so that make-mod-scripts is already built), then:
bitbake linux-yocto my-kernel-module -c clean
bitbake my-kernel-module -c installWhy can't we fix the dependency ? We chased a few of these around when the mod-scripts recipe was introduced, and this just looks like a variant we missed.
Quoted 6 lines
Right, so there *is* already a correct dependency on make-mod-scripts:do_compile. The problem is, in the scenario I gave above, that task is already done and Bitbake is not going to run it again. But in fact, the stuff it left behind in ${STAGING_KERNEL_BUILDDIR} was cleaned out when I cleaned the linux-yocto recipe.
Quoted 2 lines
Yes, and that clean is supposed to invalidate things, and trigger the re-run.
Hi Dan,
Are you seeing this on master, or some other branch ?
I can run the following on master, as often as I like:
% bitbake linux-yocto hello-mod -c clean; bitbake hello-mod -c install
And I see the required scripts coming and going from the build artifacts as expected:
build [/home/bruc...-artifacts]> find . -name 'fixdep' ./tools/objtool/fixdep./scripts/basic/fixdep
# after the clean: build [/home/bruc...-artifacts]> find . -name 'fixdep'
# after the install build [/home/bruc...-artifacts]> cd .. build [/home/bruc...qemux86-64]> cd - /home/bruce/poky/build/tmp/work-shared/qemux86-64/kernel-build-artifacts build [/home/bruc...-artifacts]> find . -name 'fixdep' ./tools/objtool/fixdep ./scripts/basic/fixdep
Is there something different with your module recipe than the one in meta-skeleton ? Can you share it, so I can debug further ?
Bruce
Quoted 26 lines
Quoted 6 lines
I guess the real issue is that we have this "shared" work directory which a bunch of different recipes contribute files to, but can be cleaned out by running the "clean" task on any one of several different recipes.
We've solved that issue in the past, I'll have a look on Wednesday to see what may have changed and (re)broken it. There may be a bugzilla floating around for that, I'll see if I can track it down as well.
Bruce
Quoted 5 lines
--
Dan Callaghan <[email protected]>
Software Engineer
Opengear <https://opengear.com/>--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Excerpts from Bruce Ashfield's message of 2020-04-22 13:40:33 -04:00:
Quoted 1 line
Are you seeing this on master, or some other branch ?
Unfortunately we are still on Warrior, and I can't easily test any newer branches right now.
Quoted 14 lines
I can run the following on master, as often as I like:
% bitbake linux-yocto hello-mod -c clean; bitbake hello-mod -c install
And I see the required scripts coming and going from the build artifacts as expected:
build [
/home/bruc...-artifacts]> find . -name 'fixdep'./tools/objtool/fixdep./scripts/basic/fixdep# after the clean: build [/home/bruc...-artifacts]> find . -name 'fixdep'
Quoted 7 lines
# after the install build [/home/bruc...-artifacts]> cd .. build [/home/bruc...qemux86-64]> cd - /home/bruce/poky/build/tmp/work-shared/qemux86-64/kernel-build-artifacts build [/home/bruc...-artifacts]> find . -name 'fixdep' ./tools/objtool/fixdep ./scripts/basic/fixdep
Okay, that's interesting. Thanks for testing it.
When you do install on the module, does Bitbake somehow know it must also re-run make-mod-scripts:do_compile, or is fixdep getting put into the kernel-build-artifacts directory by something else?
Quoted 2 lines
Is there something different with your module recipe than the one in meta-skeleton ? Can you share it, so I can debug further ?
We see this with all our out-of-tree kernel module recipes and they are all pretty ordinary, just using module.bbclass. Maybe there is something strange we have done with our kernel recipe which is causing it.
I can't share the entire layer but here is an example of one of the module recipes:
https://gist.github.com/danc86/09143b04f43e47f99ae91bbdee53bf2f
I'll just paste the entire thing here too since it is so short:
SUMMARY = "nct5104d watchdog driver"
DESCRIPTION = "Linux driver for NCT5104D watchdog function"
HOMEPAGE = "https://github.com/opengear/nct5104d-watchdog-driver" LICENSE = "GPLv2"LIC_FILES_CHKSUM = "file://LICENSE;md5=b234ee4d69f5fce4486a80fdaf4a4263" SRC_URI = "git://github.com/opengear/nct5104d-watchdog-driver.git;protocol=https;" SRCREV = "5495f4eaa15bb0b8804778df046f5276d66413ce"
PR = "r1"
S = "${WORKDIR}/git"
inherit module
Signature · 3 lines
Dan Callaghan <[email protected]> Software Engineer Opengear <https://opengear.com/>