kernel: consolidated pull for developer experience
cover kernel: consolidated pull for developer experience
Richard,
Here are the first (and largest) kernel developer experience changes for the 1.8 release.
Obviously you are the author of patch 1 in the series, and I've made changes to fixup the remaining out of tree module build, and on-target kernel build issues.
With these changes in place, I've built and booted the linux-yocto kernels, as well as build the kernel itself on the target .. so they are definitely sane from that point of view. I won't swear that there aren't any cornder cases still lurking, but we need more eyes and tests to find those cases.
With this series, we've move a lot of the source copying and I/O intensive operations to the kernelsrc recipe .. which makes them optional and not an impact on the typical build/developer. As described in pach 1, the kernel is unpacked, manipulated and built directly from STAGING_DIR_KERNEL.
If the src is required, the kerneldevsrc recipe can be added to the DEPENDS/RDEPENDS and the packaging will happen (with the associated overhead).
I've kept the series history as you see it on purpose, so we can capture credit for the optimization idea, and to make the changes more digestible to those reading the series. If we want some additional squashing of the commits, I'm happy to do that as well.
The final parts of this series are some collected kernel build fixes that I've had for a while, but are minor in the grand scheme of things.
Cheers,
Bruce
Bruce Ashfield (5):
kernel: fix out of tree module builds
kerneldev: create kernel-devsrc packaging
images: introduce core-image-kernel-dev
lttng/perf: depend on virtual/kernel:do_install
kernel-yocto: fix non-git builds
Jeff Wang (1):
kernel-yocto: make sure git tags get dereferenced properly in
do_patch()
Richard Purdie (1):
kernel: Rearrange for 1.8
meta/classes/kernel-yocto.bbclass | 8 +- meta/classes/kernel.bbclass | 93 ++++------------------ meta/classes/linux-kernel-base.bbclass | 13 ++- .../images/core-image-kernel-dev.bb | 17 ++++ meta/recipes-kernel/linux/kernel-devsrc.bb | 50 ++++++++++++ meta/recipes-kernel/lttng/lttng-modules_2.5.0.bb | 1 + meta/recipes-kernel/perf/perf.bb | 12 +-- 7 files changed, 104 insertions(+), 90 deletions(-) create mode 100644 meta/recipes-extended/images/core-image-kernel-dev.bb create mode 100644 meta/recipes-kernel/linux/kernel-devsrc.bb
Signature · 1 line
2.1.0
PATCH 1/7 kernel: Rearrange for 1.8
From: Richard Purdie <[email protected]>
In 1.8 we want to streamline the kernel build process. Basically we currently have multiple copies of the kernel source floating around and the copying/compression/decompression is painful.
Lets assume we have a kernel source per machine since in most cases this is true (and we have a sysroot per machine anyway). Basically, instead of extracting a source into WORKDIR, then copying to a sysroot, we now set S to point straight at STAGING_DIR_KERNEL.
Anything using kernel source can then just point at it and use:
do_configure[depends] += "virtual/kernel:do_patch"
to depend on the kernel source being present. Note this is different behaviour to DEPENDS += "virtual/kernel" which equates to do_configure[depends] += "virtual/kernel:do_populate_sysroot".
Once we do this, we no longer need the copy operation in do_populate_sysroot, in fact there is nothing to do there (yay).
The remaining part of the challenge is to kill off the horrible do_install. This patch splits it off to a different class, the idea here is to have a separate recipe which depends on the virtual/kernel:do_patch and just installs and packages the source needed to build modules on target into a specific package.
Right now this code is proof of concept. It builds kernels and kernel modules. perf blows up in do_package with issues on finding the kernel version which can probably be fixed by adding back the right bit of do_install, and adding a dependency of do_package[depends] += "virtual/kernel:do_install" to perf. The whole thing needs a good write up, the corner cases testing and probably a good dose of cleanup to the remaining code.
--- meta/classes/kernel-yocto.bbclass | 2 - meta/classes/kernel.bbclass | 91 ++-------------------------------- meta/classes/linux-kernel-base.bbclass | 13 ++++- meta/recipes-kernel/perf/perf.bb | 12 ++--- 4 files changed, 20 insertions(+), 98 deletions(-)
meta/classes/kernel-yocto.bbclass +0 -2
@@ -1,5 +1,3 @@ -S = "${WORKDIR}/linux" - # remove tasks that modify the source tree in case externalsrc is inherited SRCTREECOVEREDTASKS += "do_kernel_link_vmlinux do_kernel_configme do_validate_branches do_kernel_configcheck do_kernel_checkout do_patch"
meta/classes/kernel.bbclass +3 -88
109 lines not on this page · raw message
meta/classes/linux-kernel-base.bbclass +11 -2
@@ -1,5 +1,5 @@ # parse kernel ABI version out of <linux/version.h> -def get_kernelversion(p): +def get_kernelversion_headers(p): import re fn = p + '/include/linux/utsrelease.h'
@@ -9,7 +9,6 @@ def get_kernelversion(p): if not os.path.isfile(fn): fn = p + '/include/linux/version.h' - import re try: f = open(fn, 'r') except IOError:
@@ -24,6 +23,16 @@ def get_kernelversion(p): return m.group(1) return None + +def get_kernelversion_file(p): + fn = p + '/kernel-abiversion' + + try: + with open(fn, 'r') as f: + return f.readlines()[0].strip() + except IOError: + return None + def linux_module_packages(s, d): suffix = "" return " ".join(map(lambda s: "kernel-module-%s%s" % (s.lower().replace('_', '-').replace('@', '+'), suffix), s.split()))
meta/recipes-kernel/perf/perf.bb +6 -6
@@ -22,7 +22,7 @@ TUI_DEPENDS = "${@perf_feature_enabled('perf-tui', 'libnewt slang', '',d)}" SCRIPTING_DEPENDS = "${@perf_feature_enabled('perf-scripting', 'perl python', '',d)}" LIBUNWIND_DEPENDS = "${@perf_feature_enabled('perf-libunwind', 'libunwind', '',d)}" -DEPENDS = "virtual/kernel \ +DEPENDS = " \ virtual/${MLPREFIX}libc \ ${MLPREFIX}elfutils \ ${MLPREFIX}binutils \
@@ -32,6 +32,8 @@ DEPENDS = "virtual/kernel \ bison flex \ " +do_configure[depends] += "virtual/kernel:do_patch" + PROVIDES = "virtual/perf" inherit linux-kernel-base kernel-arch pythonnative
@@ -46,7 +48,7 @@ export HOST_SYS #kernel 3.1+ supports WERROR to disable warnings as errors export WERROR = "0" -do_populate_lic[depends] += "virtual/kernel:do_populate_sysroot" +do_populate_lic[depends] += "virtual/kernel:do_patch" # needed for building the tools/perf Perl binding inherit perlnative cpan-base
@@ -56,9 +58,7 @@ export PERL_INC = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)} export PERL_LIB = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}" export PERL_ARCHLIB = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}" -S = "${STAGING_KERNEL_DIR}" -# The source should be ready after the do_unpack -do_unpack[depends] += "virtual/kernel:do_populate_sysroot" +inherit kernelsrc B = "${WORKDIR}/${BPN}-${PV}"
@@ -157,7 +157,7 @@ do_configure_prepend () { } python do_package_prepend() { - bb.data.setVar('PKGV', get_kernelversion('${S}').split("-")[0], d) + d.setVar('PKGV', d.getVar("KERNEL_VERSION", True).split("-")[0]) } PACKAGE_ARCH = "${MACHINE_ARCH}"
Signature · 1 line
2.1.0
Re: [PATCH 1/7] kernel: Rearrange for 1.8
Quoted 237 lines · Bruce Ashfield
From: Richard Purdie <[email protected]>
In 1.8 we want to streamline the kernel build process. Basically we currently have multiple copies of the kernel source floating around and the copying/compression/decompression is painful.
Lets assume we have a kernel source per machine since in most cases this is true (and we have a sysroot per machine anyway). Basically, instead of extracting a source into WORKDIR, then copying to a sysroot, we now set S to point straight at
STAGING_DIR_KERNEL.Anything using kernel source can then just point at it and use:
do_configure[depends] += "virtual/kernel:do_patch"
to depend on the kernel source being present. Note this is different behaviour to DEPENDS += "virtual/kernel" which equates to do_configure[depends] += "virtual/kernel:do_populate_sysroot".
Once we do this, we no longer need the copy operation in
do_populate_sysroot, in fact there is nothing to do there (yay).The remaining part of the challenge is to kill off the horrible
do_install. This patch splits it off to a different class, the idea here is to have a separate recipe which depends on the virtual/kernel:do_patch and just installs and packages the source needed to build modules on target into a specific package.Right now this code is proof of concept. It builds kernels and kernel modules. perf blows up in
do_packagewith issues on finding the kernel version which can probably be fixed by adding back the right bit ofdo_install, and adding a dependency of do_package[depends] += "virtual/kernel:do_install" to perf. The whole thing needs a good write up, the corner cases testing and probably a good dose of cleanup to the remaining code.Signed-off-by Richard Purdie <[email protected]>--- meta/classes/kernel-yocto.bbclass | 2 - meta/classes/kernel.bbclass | 91 ++-------------------------------- meta/classes/linux-kernel-base.bbclass | 13 ++++- meta/recipes-kernel/perf/perf.bb | 12 ++--- 4 files changed, 20 insertions(+), 98 deletions(-)meta/classes/kernel-yocto.bbclass +0
-2@@ -1,5 +1,3 @@ -S = "${WORKDIR}/linux" - # remove tasks that modify the source tree in case externalsrc is inherited SRCTREECOVEREDTASKS += "do_kernel_link_vmlinux do_kernel_configme do_validate_branches do_kernel_configcheck do_kernel_checkout do_patch"meta/classes/kernel.bbclass +3
-88@@ -3,6 +3,8 @@ inherit linux-kernel-base kernel-module-split PROVIDES += "virtual/kernel" DEPENDS += "virtual/${TARGET_PREFIX}binutils virtual/${TARGET_PREFIX}gcc kmod-native depmodwrapper-cross bc-native" +S = "${STAGING_DIR_TARGET}/${KERNEL_SRC_PATH}" + # we include gcc above, we dont need virtual/libc INHIBIT_DEFAULT_DEPS = "1"@@ -55,7 +57,7 @@ KERNEL_IMAGEDEST = "boot" # export CMDLINE_CONSOLE = "console=${@d.getVar("KERNEL_CONSOLE",1) or "ttyS0"}" -KERNEL_VERSION = "${@get_kernelversion('${B}')}" +KERNEL_VERSION = "${@get_kernelversion_headers('${B}')}" KERNEL_LOCALVERSION ?= ""@@ -221,93 +223,6 @@ kernel_do_install() { # echo "${KERNEL_VERSION}" > $kerneldir/kernel-abiversion - - # - # Store kernel image name to allow use during image generation - # - - echo "${KERNEL_IMAGE_BASE_NAME}" >$kerneldir/kernel-image-name - - # - # Copy the entire source tree. In case an external build directory is - # used, copy the build directory over first, then copy over the source - # dir. This ensures the original Makefiles are used and not the - # redirecting Makefiles in the build directory. - # - find . -depth -not -name "*.cmd" -not -name "*.o" -not -name "*.so.dbg" -not -name "*.so" -not -path "./Documentation*" -not -path "./source*" -not -path "./.*" -print0 | cpio --null -pdlu $kerneldir - cp .config $kerneldir - if [ "${S}" != "${B}" ]; then - pwd="$PWD" - cd "${S}" - find . -depth -not -path "./Documentation*" -not -path "./.*" -print0 | cpio --null -pdlu $kerneldir - cd "$pwd" - fi - - # Test to ensure that the output file and image type are not actually - # the same file. If hardlinking is used, they will be the same, and there's - # no need to install. - ! [ ${KERNEL_OUTPUT} -ef $kerneldir/${KERNEL_IMAGETYPE} ] && install -m 0644 ${KERNEL_OUTPUT} $kerneldir/${KERNEL_IMAGETYPE} - install -m 0644 System.map $kerneldir/System.map-${KERNEL_VERSION} - - # Dummy Makefile so the clean below works - mkdir $kerneldir/Documentation - touch $kerneldir/Documentation/Makefile - - # - # Clean and remove files not needed for building modules. - # Some distributions go through a lot more trouble to strip out - # unecessary headers, for now, we just prune the obvious bits. - # - # We don't want to leave host-arch binaries in /sysroots, so - # we clean the scripts dir while leaving the generated config - # and include files. - # - oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean _mrproper_scripts - - # hide directories that shouldn't have their .c, s and S files deleted - for d in tools scripts lib; do - mv $kerneldir/$d $kerneldir/.$d - done - - # delete .c, .s and .S files, unless we hid a directory as .<dir>. This technique is - # much faster than find -prune and -exec - find $kerneldir -not -path '*/\.*' -type f -name "*.[csS]" -delete - - # put the hidden dirs back - for d in tools scripts lib; do - mv $kerneldir/.$d $kerneldir/$d - done - - # As of Linux kernel version 3.0.1, the clean target removes - # arch/powerpc/lib/crtsavres.o which is present in - # KBUILD_LDFLAGS_MODULE, making it required to build external modules. - if [ ${ARCH} = "powerpc" ]; then - cp -l arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o - fi - - # Necessary for building modules like compat-wireless. - if [ -f include/generated/bounds.h ]; then - cp -l include/generated/bounds.h $kerneldir/include/generated/bounds.h - fi - if [ -d arch/${ARCH}/include/generated ]; then - mkdir -p $kerneldir/arch/${ARCH}/include/generated/ - cp -flR arch/${ARCH}/include/generated/* $kerneldir/arch/${ARCH}/include/generated/ - fi - - # Remove the following binaries which cause strip or arch QA errors - # during do_package for cross-compiled platforms - bin_files="arch/powerpc/boot/addnote arch/powerpc/boot/hack-coff \ - arch/powerpc/boot/mktree scripts/kconfig/zconf.tab.o \ - scripts/kconfig/conf.o scripts/kconfig/kxgettext.o" - for entry in $bin_files; do - rm -f $kerneldir/$entry - done - - # kernels <2.6.30 don't have $kerneldir/tools directory so we check if it exists before calling sed - if [ -f $kerneldir/tools/perf/Makefile ]; then - # Fix SLANG_INC for slang.h - sed -i 's#-I/usr/include/slang#-I=/usr/include/slang#g' $kerneldir/tools/perf/Makefile - fi } do_install[prefuncs] += "package_get_auto_pr"meta/classes/linux-kernel-base.bbclass +11
-2@@ -1,5 +1,5 @@ # parse kernel ABI version out of <linux/version.h> -def get_kernelversion(p): +def get_kernelversion_headers(p): import re fn = p + '/include/linux/utsrelease.h'@@ -9,7 +9,6 @@ def get_kernelversion(p): if not os.path.isfile(fn): fn = p + '/include/linux/version.h' - import re try: f = open(fn, 'r') except IOError:@@ -24,6 +23,16 @@ def get_kernelversion(p): return m.group(1) return None + +def get_kernelversion_file(p): + fn = p + '/kernel-abiversion' + + try: + with open(fn, 'r') as f: + return f.readlines()[0].strip() + except IOError: + return None + def linux_module_packages(s, d): suffix = "" return " ".join(map(lambda s: "kernel-module-%s%s" % (s.lower().replace('_', '-').replace('@', '+'), suffix), s.split()))meta/recipes-kernel/perf/perf.bb +4
-2@@ -22,7 +22,7 @@ TUI_DEPENDS = "${@perf_feature_enabled('perf-tui', 'libnewt slang', '',d)}" SCRIPTING_DEPENDS = "${@perf_feature_enabled('perf-scripting', 'perl python', '',d)}" LIBUNWIND_DEPENDS = "${@perf_feature_enabled('perf-libunwind', 'libunwind', '',d)}" -DEPENDS = "virtual/kernel \ +DEPENDS = " \ virtual/${MLPREFIX}libc \ ${MLPREFIX}elfutils \ ${MLPREFIX}binutils \@@ -32,6 +32,8 @@ DEPENDS = "virtual/kernel \ bison flex \ " +do_configure[depends] += "virtual/kernel:do_patch" + PROVIDES = "virtual/perf" inherit linux-kernel-base kernel-arch pythonnative@@ -46,7 +48,7 @@ export HOST_SYS #kernel 3.1+ supports WERROR to disable warnings as errors export WERROR = "0" -do_populate_lic[depends] += "virtual/kernel:do_populate_sysroot" +do_populate_lic[depends] += "virtual/kernel:do_patch" # needed for building the tools/perf Perl binding inherit perlnative cpan-base
meta/recipes-kernel/perf/perf.bb export PERL_INC = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)} · line 61
Quoted 7 lines
meta/recipes-kernel/perf/perf.bb +1
-3@@ -56,9 +58,7 @@ export PERL_INC = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)} export PERL_LIB = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}" export PERL_ARCHLIB = "${STAGING_LIBDIR}${PERL_OWN_DIR}/perl/${@get_perl_version(d)}" -S = "${STAGING_KERNEL_DIR}" -# The source should be ready after the do_unpack -do_unpack[depends] += "virtual/kernel:do_populate_sysroot" +inherit kernelsrc
When juggling the patches .. I managed to leave this in .. when it should have been removed. I'm fixing up this patch to no longer inherit this class, since
it is no longer needed.
Bruce
Quoted 20 lines
meta/recipes-kernel/perf/perf.bb +1
-2B = "${WORKDIR}/${BPN}-${PV}"@@ -157,7 +157,7 @@ do_configure_prepend () { } python do_package_prepend() { - bb.data.setVar('PKGV', get_kernelversion('${S}').split("-")[0], d) + d.setVar('PKGV', d.getVar("KERNEL_VERSION", True).split("-")[0]) } PACKAGE_ARCH = "${MACHINE_ARCH}" --2.1.0
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"
PATCH 2/7 kernel: fix out of tree module builds
In reply to Bruce AshfieldWith the kernel build optimizations, we no longer copy the source from the built kernel into the staging dir, since the kernel is unpacked and built directly from the staging dir.
This means that a few build artifacts need to be restored to allow out of tree modules to once again build.
--- meta/classes/kernel.bbclass | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
meta/classes/kernel.bbclass +22 -0
@@ -223,6 +223,28 @@ kernel_do_install() { # echo "${KERNEL_VERSION}" > $kerneldir/kernel-abiversion + + # Copy files required for module builds + cp System.map $kerneldir/System.map-${KERNEL_VERSION} + cp Module.symvers $kerneldir/ + cp .config $kerneldir/ + mkdir -p $kerneldir/include/config + cp include/config/kernel.release $kerneldir/include/config/kernel.release + + # As of Linux kernel version 3.0.1, the clean target removes + # arch/powerpc/lib/crtsavres.o which is present in + # KBUILD_LDFLAGS_MODULE, making it required to build external modules. + if [ ${ARCH} = "powerpc" ]; then + cp arch/powerpc/lib/crtsavres.o $kerneldir/arch/powerpc/lib/crtsavres.o + fi + + mkdir -p $kerneldir/include/generated/ + cp -fR include/generated/* $kerneldir/include/generated/ + + if [ -d arch/${ARCH}/include/generated ]; then + mkdir -p $kerneldir/arch/${ARCH}/include/generated/ + cp -fR arch/${ARCH}/include/generated/* $kerneldir/arch/${ARCH}/include/generated/ + fi } do_install[prefuncs] += "package_get_auto_pr"
Signature · 1 line
2.1.0
Re: [PATCH 2/7] kernel: fix out of tree module builds
Bruce Ashfield <[email protected]>
meta/classes/kernel.bbclass kernel_do_install() { · line 230
Quoted 12 lines · writes:
With the kernel build optimizations, we no longer copy the source from the built kernel into the staging dir, since the kernel is unpacked and built directly from the staging dir.
This means that a few build artifacts need to be restored to allow out of tree modules to once again build.
...meta/classes/kernel.bbclass +5
-0+ + # Copy files required for module builds + cp System.map $kerneldir/System.map-${KERNEL_VERSION} + cp Module.symvers $kerneldir/ + cp .config $kerneldir/
This and the following lines seem to void the whole idea of the patchset because it means that only one kernel can be built. In our usecase we build usually two kernels: a full featured one for the main system and a minimal, initramfs based for a rescue system.
With this change and building both kernels, it is ambiguous which System.map is used.
Enrico
Signature · 3 lines
SIGMA Chemnitz GmbH Registergericht: Amtsgericht Chemnitz HRB 1750 Am Erlenwald 13 Geschaeftsfuehrer: Grit Freitag, Frank Pyritz 09128 Chemnitz
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 21 lines · Enrico Scholz
Bruce Ashfield <[email protected]>
meta/classes/kernel.bbclass kernel_do_install() { · line 230
Quoted 12 lines · writes:
With the kernel build optimizations, we no longer copy the source from the built kernel into the staging dir, since the kernel is unpacked and built directly from the staging dir.
This means that a few build artifacts need to be restored to allow out of tree modules to once again build.
...meta/classes/kernel.bbclass +5
-0+ + # Copy files required for module builds + cp System.map $kerneldir/System.map-${KERNEL_VERSION} + cp Module.symvers $kerneldir/ + cp .config $kerneldir/This and the following lines seem to void the whole idea of the patchset because it means that only one kernel can be built. In our usecase we build usually two kernels: a full featured one for the main system and a minimal, initramfs based for a rescue system.
Quoted 2 lines
With this change and building both kernels, it is ambiguous which System.map is used.
Regardless of how you do this, if you build two kernels, you still need to choose one to build the main set of modules against. I'm therefore not sure how this change makes anything particularly worst than what existed before?
Building two kernels is currently fraught with difficulty anyway, to the point of not being a currently well supported use case and the classes would probably need to be refactored to allow such things to be well supported. These changes do bring significant benefit to the most common case users so we probably need to find a way to allow your use case to work with the new model.
Cheers,
Richard
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 10 lines · Richard Purdie
Quoted 5 lines
In our usecase we build usually two kernels: a full featured one for the main system and a minimal, initramfs based for a rescue system.
With this change and building both kernels, it is ambiguous which System.map is used.
Regardless of how you do this, if you build two kernels, you still need to choose one to build the main set of modules against. I'm therefore not sure how this change makes anything particularly worst than what existed before?
A I see... previously I did
| do_install[noexec] = "1"
in the rescue kernel and only the main kernel's System.map, .config... were staged into the sysroot. This can be done now too, but building secondary kernels fails with
| .../usr/src/kernel is not clean, please run 'make mrproper'
Quoted 1 line
Building two kernels is currently fraught with difficulty anyway,
It was not so difficulty... Removing the "virtual/kernel" provider and changing names of deployed files should do it with the old classes.
Quoted 3 lines
to the point of not being a currently well supported use case and the classes would probably need to be refactored to allow such things to be well supported. These changes do bring significant benefit
Does some lowered i/o really provide such a benefit? With the new system you are bypassing the excellent staging system (which would detect e g ".config" or "System.map" file conflicts). Afais, it breaks with "rm_work" too (I guess, kernel:fetch stamp is removed when kernel build finishes and packages which inherit "kernelsrc" fetch the kernel sources (without .config and System.map) again).
Enrico
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 11 lines · Enrico Scholz
Quoted 3 lines · Richard Purdie
to the point of not being a currently well supported use case and the classes would probably need to be refactored to allow such things to be well supported. These changes do bring significant benefit
Does some lowered i/o really provide such a benefit? With the new system you are bypassing the excellent staging system (which would detect e g ".config" or "System.map" file conflicts). Afais, it breaks with "
rm_work" too (I guess, kernel:fetch stamp is removed when kernel build finishes and packages which inherit "kernelsrc" fetch the kernel sources (without .config and System.map) again).
In summary, basically, yes. The kernel source is huge and we were compressing/decompressing it in several places on the critical path. People were struggling to develop kernels using the system due to the overhead.
Whilst this approach does bypass some parts of the system, I do believe the benefits are worth it. We're talking about making the kernel build time about three times faster iirc, and reducing its sstate footprint significantly.
Cheers,
Richard
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 4 lines · Richard Purdie
In summary, basically, yes. The kernel source is huge and we were compressing/decompressing it in several places on the critical path. People were struggling to develop kernels using the system due to the overhead.
I do not see how the new system makes it easier to "develop kernels". Due to polluting sources with .config and related files, KBUILD_OUTPUT/VPATH builds are not possible from this tree.
In my experience, it is also a pain to jump between the different directories and tools like 'cscope' are not prepared of it.
And more importantly: the new system lowers end user experience significantly because kernel messages will show absolute buildsystem paths now; e.g.
| /srv/.oe/bld/e6ca2c38-c20d-f57f-7eca-ffc0aaa2f0bd/sysroots/kk-trizeps6/usr/src/kernel/drivers/usb/core/hub.c
vs.
| drivers/usb/core/hub.c
VPATH builds might be interesting for QA (e.g. building from same source with different configuration) but should not be used for final builds.
Quoted 3 lines
Whilst this approach does bypass some parts of the system, I do believe the benefits are worth it. We're talking about making the kernel build time about three times faster iirc,
I can not reproduce these numbers here; I get (after a '-c cleanall' and 'ccache -c'):
| Task | time (old) | time (new) | |------------------------------+------------+------------| | do_bundle_initramfs | 0.087052 | 0.034955 | | do_compile | 128.242407 | 133.723027 | | do_compile_kernelmodules | 84.415655 | 83.249409 | | do_compile_prepare | 2.885401 | 1.714159 | | do_configure | 6.202691 | 4.340526 | | do_deploy | 13.991785 | 14.07096 | | do_fetch | 0.210244 | 1.425304 | | do_generate_initramfs_source | 0.063915 | 0.041925 | | do_install | 16.190504 | 2.91906 | | do_package | 120.823374 | 16.422429 | | do_package_qa | | 2.557622 | | do_package_write_ipk | 42.50694 | 29.57585 | | do_packagedata | 1.612542 | 0.462001 | | do_patch | 0.186583 | 0.011232 | | do_populate_lic | 0.795013 | 0.135186 | | do_populate_sysroot | 10.142978 | 0.533519 | | do_rm_work | 1.762486 | 0.447187 | | do_rm_work_all | 0.049144 | 0.030964 | | do_sizecheck | 0.054441 | 0.035806 | | do_strip | 0.049917 | 0.030841 | | do_uboot_mkimage | 9.032543 | 12.83624 | | do_unpack | 6.695678 | 9.322173 |
| old | 446.00129 | | new | 313.92038 |
Although the 'new' system is faster, this is gained mainly by the 'do_package' task which does not seem to be comparable. The new method will create only a very small 'kernel-dev' subpackage:
1,1M tmp/deploy/ipk/kk_trizeps6/kernel-dev_3.14...
vs.
36M tmp/deploy/ipk/kk_trizeps6/kernel-dev_3.14...
so the old task can be improved either by removing some files, or the new task misses files.
Enrico
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 8 lines · Enrico Scholz
Quoted 4 lines · Richard Purdie
In summary, basically, yes. The kernel source is huge and we were compressing/decompressing it in several places on the critical path. People were struggling to develop kernels using the system due to the overhead.
I do not see how the new system makes it easier to "develop kernels".
One of the complaints I keep hearing is how long the populate_sysroot and package tasks take. I also get complaints about the sstate footprint of those tasks.
Imagine you're doing a "bitbake virtual/kernel -c compile -f", hacking on a module, you then package and install it onto the target to test.
As things stood, the package and populate_sysroot tasks took a while (I'll get to times in a minute). This was particularly true with large kernel source directories.
We also had issues about how and when "make scripts" should be run on the kernel directory to enable module building and conflicts over people wanting the source code to be able to develop modules on target. Not everyone wants the latter but it should be available to those who do want it and the old kernel-dev package was basically crippled.
Taking a step back from things and keeping in mind some of the "developer experience" goals of the 1.8 release, we therefore looked at how we could improve things. The biggest problem and overhead is the amount of copying of a large chunk of data (the kernel source) that we make and the fact that it was incomplete in kernel-dev due to wanting speed (at the expense of breaking it). The changes therefore look at improving things in that area.
Quoted 3 lines
Due
to polluting sources with .config and related files, KBUILD_OUTPUT/VPATH builds are not possible from this tree.
Agreed and I'm actually not happy about this. I think going forward we'll need a second directory for these so we do keep the output separate and can recompile against it. Bruce, please take note we probably need to change this.
Quoted 7 lines
In my experience, it is also a pain to jump between the different directories and tools like 'cscope' are not prepared of it.
And more importantly: the new system lowers end user experience significantly because kernel messages will show absolute buildsystem paths now; e.g.
Quoted 5 lines
|
/srv/.oe/bld/e6ca2c38-c20d-f57f-7eca-ffc0aaa2f0bd/sysroots/kk-trizeps6/usr/src/kernel/drivers/usb/core/hub.cvs.
|
drivers/usb/core/hub.c
See my other email on the two sides to this. Here I can tell from a cut and pasted failure which revisions you were trying to build so it actually makes the logs more useful in some contexts. That isn't the reason we've done this but I'd say it can be helpful.
Quoted 7 lines
VPATH builds might be interesting for QA (e.g. building from same source with different configuration) but should not be used for final builds.
Quoted 3 lines
Whilst this approach does bypass some parts of the system, I do believe the benefits are worth it. We're talking about making the kernel build time about three times faster iirc,
I did say "iirc" and I don't exactly remember the circumstances I tested that, see below, sorry :(.
Quoted 28 lines
I can not reproduce these numbers here; I get (after a '-c cleanall' and 'ccache -c'):
| Task | time (old) | time (new) | |------------------------------+------------+------------| | do_bundle_initramfs | 0.087052 | 0.034955 | | do_compile | 128.242407 | 133.723027 | | do_compile_kernelmodules | 84.415655 | 83.249409 | | do_compile_prepare | 2.885401 | 1.714159 | | do_configure | 6.202691 | 4.340526 | | do_deploy | 13.991785 | 14.07096 | | do_fetch | 0.210244 | 1.425304 | | do_generate_initramfs_source | 0.063915 | 0.041925 | | do_install | 16.190504 | 2.91906 | | do_package | 120.823374 | 16.422429 | | do_package_qa | | 2.557622 | | do_package_write_ipk | 42.50694 | 29.57585 | | do_packagedata | 1.612542 | 0.462001 | | do_patch | 0.186583 | 0.011232 | | do_populate_lic | 0.795013 | 0.135186 | | do_populate_sysroot | 10.142978 | 0.533519 | | do_rm_work | 1.762486 | 0.447187 | | do_rm_work_all | 0.049144 | 0.030964 | | do_sizecheck | 0.054441 | 0.035806 | | do_strip | 0.049917 | 0.030841 | | do_uboot_mkimage | 9.032543 | 12.83624 | | do_unpack | 6.695678 | 9.322173 |
Quoted 2 lines
| old | 446.00129 | | new | 313.92038 |
So you have a gain here from 7.4 mins to 5.2 mins which isn't bad. I'd observe your numbers suggest you have pretty fast disk IO, I've seen the populate_sysroot task take a lot longer due to the amount of data being copied around. Keep in mind that whilst do_package improved, so did package_write_ipk, rm_work, install and populate_sysroot, all quite a bit more than "three times".
I suspect my "three times" number comes from my highly parallel core system where the do_compile/do_compilemodules step runs extremely quickly (say 20s) but do_package was about the same as this so you can imagine if you reduce do_package from 120s to 16s, you could see an increase of about "three times".
To give another perspective on the times, we run "official" benchmarks of some things and the line before and after the changes merged from that log is:
fedora19,master:88528a128fe7c11871c24706ff6d245b183d6975,1.7_M2-1510-g88528a1,1:18:03,11:16.26,1:13:35,4:11.18,0:34.68,0:15.72,0:01.08,24805964,5548960 fedora19,master:b99419ff4c1b5ac1e4acd34d95fffd3ac5458bad,1.7_M2-1553-gb99419f,1:13:20,8:10.41,1:10:07,4:08.31,0:34.67,0:15.77,0:01.05,24385880,5956872
What that shows is that:
The "bitbake core-image-sato -c rootfs" time without rm_work went from 1:18 to 1:13, i.e. improved 5 mins.
The "bitbake virtual/kernel" time went from 11:16 to 8:10, i.e. improved by 3 mins.
The "bitbake core-image-sato -c rootfs" time with rm_work went from 1:13 to 1:10, i.e. improved 3 mins.
and the size of the build with rm_work increased by around 500MB, the size of the build without rm_work decreased by 500MB.
I will also add that we're not done yet with some of these tweaks. I know linux-yocto is doing some things in do_patch that are performance intensive and we plan to fix those which will further improve the 11 mins number above.
Quoted 4 lines
Although the 'new' system is faster, this is gained mainly by the '
do_package' task which does not seem to be comparable. The new method will create only a very small 'kernel-dev' subpackage:
Quoted 8 lines
1,1M tmp/deploy/ipk/kk_trizeps6/kernel-dev_3.14...vs.
36M tmp/deploy/ipk/kk_trizeps6/kernel-dev_3.14...so the old task can be improved either by removing some files, or the new task misses files.
There is a new kernel-devsrc package which contains some things that used to be in kernel-dev. It is way more complete and functional than the old one ever was since the old one was a hacked up copy of the kernel source. It also only gets built if you request it, only a small number of people need it and its hence much more user friendly this way.
I am sorry you're not happy about the changes :(.
Cheers,
Richard
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 10 lines · Richard Purdie
Quoted 6 lines
Quoted 4 lines
In summary, basically, yes. The kernel source is huge and we were compressing/decompressing it in several places on the critical path. People were struggling to develop kernels using the system due to the overhead.
I do not see how the new system makes it easier to "develop kernels".
One of the complaints I keep hearing is how long the
populate_sysrootand package tasks take. I also get complaints about the sstate footprint of those tasks.
Perhaps, it should be possible to disable sstate'ing for certain recipes? In case of the new kernel build system, you swap the expensive sstate'ing with an expensive unpacking.
Quoted 2 lines
Imagine you're doing a "bitbake virtual/kernel -c compile -f", hacking on a module, you then package and install it onto the target to test.
1. Are there really people who are doing such things? There are much more effective ways to do kernel development with OE (you have the overhead from bitbake reading its database; do_compile() does not deploy images on TFTP server nor the modules in the NFS sysroot).
2. This usecase is not supported by the current classes:
1. the 'bitbake virtual/kernel -c compile -f' removes .config and System.map from the kernel sources (else, do_compile() will fail)
2. trying to build the module (with 'M=...')will fail because .config
+ System.map are missing in the source
Quoted 6 lines
Quoted 2 lines
to polluting sources with .config and related files, KBUILD_OUTPUT/VPATH builds are not possible from this tree.
Agreed and I'm actually not happy about this. I think going forward we'll need a second directory for these so we do keep the output separate and can recompile against it.
'cp -al' a template kernel and .config etc. into ${WORKDIR} should be pretty fast.
Enrico
Re: [PATCH 2/7] kernel: fix out of tree module builds
Quoted 18 lines · Enrico Scholz
Quoted 10 lines · Richard Purdie
Quoted 6 lines
Quoted 4 lines
In summary, basically, yes. The kernel source is huge and we were compressing/decompressing it in several places on the critical path. People were struggling to develop kernels using the system due to the overhead.
I do not see how the new system makes it easier to "develop kernels".
One of the complaints I keep hearing is how long the
populate_sysrootand package tasks take. I also get complaints about the sstate footprint of those tasks.Perhaps, it should be possible to disable sstate'ing for certain recipes? In case of the new kernel build system, you swap the expensive sstate'ing with an expensive unpacking.
Quoted 7 lines
Quoted 2 lines
Imagine you're doing a "bitbake virtual/kernel -c compile -f", hacking on a module, you then package and install it onto the target to test.
1. Are there really people who are doing such things? There are much more effective ways to do kernel development with OE (you have the overhead from bitbake reading its database;
do_compile()does not deploy images on TFTP server nor the modules in the NFS sysroot).
There are definitely people doing this. And if the build system wouldn't get in the way .. more definitely could.
Quoted 5 lines
2. This usecase is not supported by the current classes:
1. the 'bitbake virtual/kernel -c compile -f' removes .config and System.map from the kernel sources (else,
do_compile()will fail)
This one is going to be fixed with the follow up patches I mentioned in another email, since we can maintain the output artifacts separate from ${B} and build against them, that keeps us working with rm_work and keeps things light.
Quoted 3 lines
2. trying to build the module (with 'M=...')will fail because .config
+ System.map are missing in the source
#2 shouldn't happen if the module depends on the kernel's do_install (as we intended), since that puts the external module build artifacts back in place.
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
to polluting sources with .config and related files, KBUILD_OUTPUT/VPATH builds are not possible from this tree.
Agreed and I'm actually not happy about this. I think going forward we'll need a second directory for these so we do keep the output separate and can recompile against it.
'cp -al' a template kernel and .config etc. into ${WORKDIR} should be pretty fast.
With a split source and build configuration .. this shouldn't be needed. If we start copying these things back into WORKDIR .. then something is wrong.
Bruce
Quoted 8 lines
Enrico
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"
Re: [PATCH 2/7] kernel: fix out of tree module builds
In reply to Richard PurdieQuoted 35 lines · Richard Purdie
Quoted 8 lines · Enrico Scholz
Quoted 4 lines · Richard Purdie
In summary, basically, yes. The kernel source is huge and we were compressing/decompressing it in several places on the critical path. People were struggling to develop kernels using the system due to the overhead.
I do not see how the new system makes it easier to "develop kernels".
One of the complaints I keep hearing is how long the
populate_sysrootand package tasks take. I also get complaints about the sstate footprint of those tasks.Imagine you're doing a "bitbake virtual/kernel -c compile -f", hacking on a module, you then package and install it onto the target to test.
As things stood, the package and
populate_sysroottasks took a while (I'll get to times in a minute). This was particularly true with large kernel source directories.We also had issues about how and when "make scripts" should be run on the kernel directory to enable module building and conflicts over people wanting the source code to be able to develop modules on target. Not everyone wants the latter but it should be available to those who do want it and the old kernel-dev package was basically crippled.
Taking a step back from things and keeping in mind some of the "developer experience" goals of the 1.8 release, we therefore looked at how we could improve things. The biggest problem and overhead is the amount of copying of a large chunk of data (the kernel source) that we make and the fact that it was incomplete in kernel-dev due to wanting speed (at the expense of breaking it). The changes therefore look at improving things in that area.
Quoted 8 lines
Quoted 3 lines
Due
to polluting sources with .config and related files, KBUILD_OUTPUT/VPATH builds are not possible from this tree.Agreed and I'm actually not happy about this. I think going forward we'll need a second directory for these so we do keep the output separate and can recompile against it. Bruce, please take note we probably need to change this.
Noted. And I have a patch in flight for this. I started it before leaving for the holidays, but with the other build issues, I didn't get a chance to complete it. This is priority #1 either over the holidays (unlikely, but I can hope) or the first weeks of January.
Bruce
Quoted 131 lines
Quoted 8 lines
In my experience, it is also a pain to jump between the different directories and tools like 'cscope' are not prepared of it.
And more importantly: the new system lowers end user experience significantly because kernel messages will show absolute buildsystem paths now; e.g.
Quoted 5 lines
|
/srv/.oe/bld/e6ca2c38-c20d-f57f-7eca-ffc0aaa2f0bd/sysroots/kk-trizeps6/usr/src/kernel/drivers/usb/core/hub.cvs.
|
drivers/usb/core/hub.cSee my other email on the two sides to this. Here I can tell from a cut and pasted failure which revisions you were trying to build so it actually makes the logs more useful in some contexts. That isn't the reason we've done this but I'd say it can be helpful.
Quoted 7 lines
VPATH builds might be interesting for QA (e.g. building from same source with different configuration) but should not be used for final builds.
Quoted 3 lines
Whilst this approach does bypass some parts of the system, I do believe the benefits are worth it. We're talking about making the kernel build time about three times faster iirc,
I did say "iirc" and I don't exactly remember the circumstances I tested that, see below, sorry :(.
Quoted 28 lines
I can not reproduce these numbers here; I get (after a '-c cleanall' and 'ccache -c'):
| Task | time (old) | time (new) | |------------------------------+------------+------------| | do_bundle_initramfs | 0.087052 | 0.034955 | | do_compile | 128.242407 | 133.723027 | | do_compile_kernelmodules | 84.415655 | 83.249409 | | do_compile_prepare | 2.885401 | 1.714159 | | do_configure | 6.202691 | 4.340526 | | do_deploy | 13.991785 | 14.07096 | | do_fetch | 0.210244 | 1.425304 | | do_generate_initramfs_source | 0.063915 | 0.041925 | | do_install | 16.190504 | 2.91906 | | do_package | 120.823374 | 16.422429 | | do_package_qa | | 2.557622 | | do_package_write_ipk | 42.50694 | 29.57585 | | do_packagedata | 1.612542 | 0.462001 | | do_patch | 0.186583 | 0.011232 | | do_populate_lic | 0.795013 | 0.135186 | | do_populate_sysroot | 10.142978 | 0.533519 | | do_rm_work | 1.762486 | 0.447187 | | do_rm_work_all | 0.049144 | 0.030964 | | do_sizecheck | 0.054441 | 0.035806 | | do_strip | 0.049917 | 0.030841 | | do_uboot_mkimage | 9.032543 | 12.83624 | | do_unpack | 6.695678 | 9.322173 |Quoted 2 lines
| old | 446.00129 | | new | 313.92038 |So you have a gain here from 7.4 mins to 5.2 mins which isn't bad. I'd observe your numbers suggest you have pretty fast disk IO, I've seen the
populate_sysroottask take a lot longer due to the amount of data being copied around. Keep in mind that whilstdo_packageimproved, so didpackage_write_ipk,rm_work, install andpopulate_sysroot, all quite a bit more than "three times".I suspect my "three times" number comes from my highly parallel core system where the do_compile/do_compilemodules step runs extremely quickly (say 20s) but
do_packagewas about the same as this so you can imagine if you reducedo_packagefrom 120s to 16s, you could see an increase of about "three times".To give another perspective on the times, we run "official" benchmarks of some things and the line before and after the changes merged from that log is:
fedora19,master:88528a128fe7c11871c24706ff6d245b183d6975,1.7_M2-1510-g88528a1,1:18:03,11:16.26,1:13:35,4:11.18,0:34.68,0:15.72,0:01.08,24805964,5548960 fedora19,master:b99419ff4c1b5ac1e4acd34d95fffd3ac5458bad,1.7_M2-1553-gb99419f,1:13:20,8:10.41,1:10:07,4:08.31,0:34.67,0:15.77,0:01.05,24385880,5956872
What that shows is that:
The "bitbake core-image-sato -c rootfs" time without
rm_workwent from 1:18 to 1:13, i.e. improved 5 mins.The "bitbake virtual/kernel" time went from 11:16 to 8:10, i.e. improved by 3 mins.
The "bitbake core-image-sato -c rootfs" time with
rm_workwent from 1:13 to 1:10, i.e. improved 3 mins.and the size of the build with
rm_workincreased by around 500MB, the size of the build withoutrm_workdecreased by 500MB.I will also add that we're not done yet with some of these tweaks. I know linux-yocto is doing some things in
do_patchthat are performance intensive and we plan to fix those which will further improve the 11 mins number above.Quoted 4 lines
Although the 'new' system is faster, this is gained mainly by the '
do_package' task which does not seem to be comparable. The new method will create only a very small 'kernel-dev' subpackage:Quoted 8 lines
1,1M tmp/deploy/ipk/kk_trizeps6/kernel-dev_3.14...vs.
36M tmp/deploy/ipk/kk_trizeps6/kernel-dev_3.14...so the old task can be improved either by removing some files, or the new task misses files.
There is a new kernel-devsrc package which contains some things that used to be in kernel-dev. It is way more complete and functional than the old one ever was since the old one was a hacked up copy of the kernel source. It also only gets built if you request it, only a small number of people need it and its hence much more user friendly this way.
I am sorry you're not happy about the changes :(.
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 2 lines
"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"
PATCH 3/7 kerneldev: create kernel-devsrc packaging
In reply to Bruce Ashfieldkernel-devsrc is responsible for creating and a packaging an environment appropriate for kernel development (on or off target).
To create this support, we only need to copy/install the results of the virtual/kernel providers build in the staging dir ... with some minor manipulations to the source tree (.git removal and a clean up). This produces a source tree that is capable of rebuilding the kernel on the target.
Installing the kernel-devsrc package on a target (along with a toolchain) is all that remains to be done.
$ cd /usr/src/kernel $ make oldconfig $ make -j2 bzImage
--- meta/recipes-kernel/linux/kernel-devsrc.bb | 50 ++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 meta/recipes-kernel/linux/kernel-devsrc.bb
meta/recipes-kernel/linux/kernel-devsrc.bb new file +50 -0
@@ -0,0 +1,50 @@ +SUMMARY = "Linux kernel Development Source" +DESCRIPTION = "Development source linux kernel. When built, this recipe packages the \ +source of the preferred virtual/kernel provider and makes it available for full kernel \ +development or external module builds" + +SECTION = "kernel" + +LICENSE = "GPLv2" +LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6" + +inherit linux-kernel-base + +# We need the kernel to be staged (unpacked, patched and configured) before +# we can grab the source and make the kernel-devsrc package +do_install[depends] += "virtual/kernel:do_populate_sysroot" + +# There's nothing to do here, except install the source where we can package it +do_fetch[noexec] = "1" +do_unpack[noexec] = "1" +do_patch[noexec] = "1" +do_configure[noexec] = "1" +do_compile[noexec] = "1" +do_populate_sysroot[noexec] = "1" + +# Define where the kernel headers are installed on the target as well as where +# they are staged. +KERNEL_SRC_PATH = "/usr/src/kernel" +S = "${STAGING_DIR_TARGET}/${KERNEL_SRC_PATH}" + +KERNEL_VERSION = "${@get_kernelversion_headers('${S}')}" + +PACKAGE_ARCH = "${MACHINE_ARCH}" + +do_install() { + kerneldir=${D}${KERNEL_SRC_PATH} + install -d $kerneldir + + # + # Copy the staging dir source (and module build support) into the devsrc structure. + # We can keep this copy simple and take everything, since a we'll clean up any build + # artifacts afterwards, and the extra i/o is not significant + # + cd ${S} + find . -type d -name '.git*' -prune -o -type f -print0 | cpio --null -pdlu $kerneldir + oe_runmake -C $kerneldir CC="${KERNEL_CC}" LD="${KERNEL_LD}" clean _mrproper_scripts +} + +PACKAGES = "kernel-devsrc" +FILES_${PN} = "${KERNEL_SRC_PATH}" +RDEPENDS_${PN} = "bc"
Signature · 1 line
2.1.0
PATCH 4/7 images: introduce core-image-kernel-dev
In reply to Bruce AshfieldWhen building a new kernel, or testing and update to the kernel there are a set of packages that are sensitive to APIs, build system and other changes associated with the kernel.
After building this recipe, we can be reasonably sure that a new kernel package and coupled userspace have been built, installed and tested via a single image.
--- meta/recipes-extended/images/core-image-kernel-dev.bb | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 meta/recipes-extended/images/core-image-kernel-dev.bb
meta/recipes-extended/images/core-image-kernel-dev.bb new file +17 -0
@@ -0,0 +1,17 @@ +DESCRIPTION = "A development image that builds the kernel and packages that are \ +sensitive to kernel updates and version changes" + +# Could also be core-image-basic, but we'll keep this small for now +require recipes-core/images/core-image-minimal.bb + +KERNEL_DEV_UTILS ?= "dropbear" +KERNEL_DEV_TOOLS ?= "packagegroup-core-tools-profile packagegroup-core-buildessential kernel-devsrc" +KERNEL_DEV_MODULE ?= "" + +CORE_IMAGE_EXTRA_INSTALL += "${KERNEL_DEV_MODULE} \ + ${KERNEL_DEV_UTILS} \ + ${KERNEL_DEV_TOOLS} \ + " + +# We need extra space for things like kernel builds, etc. +IMAGE_ROOTFS_EXTRA_SPACE_append += "+ 3000000"
Signature · 1 line
2.1.0
PATCH 5/7 lttng/perf: depend on virtual/kernel:do_install
In reply to Bruce AshfieldTo ensure that the minimal environment has been created for out of tree module builds, perf and lttng should depend on the do_install task of the kernel.
--- meta/recipes-kernel/lttng/lttng-modules_2.5.0.bb | 1 + meta/recipes-kernel/perf/perf.bb | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
meta/recipes-kernel/lttng/lttng-modules_2.5.0.bb +1 -0
@@ -7,6 +7,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=1412caf5a1aa90d6a48588a4794c0eac \ file://lgpl-2.1.txt;md5=243b725d71bb5df4a1e5920b344b86ad" DEPENDS = "virtual/kernel" +do_configure[depends] += "virtual/kernel:do_install" inherit module
meta/recipes-kernel/perf/perf.bb +1 -1
@@ -32,7 +32,7 @@ DEPENDS = " \ bison flex \ " -do_configure[depends] += "virtual/kernel:do_patch" +do_configure[depends] += "virtual/kernel:do_install" PROVIDES = "virtual/perf"
Signature · 1 line
2.1.0
PATCH 6/7 kernel-yocto: fix non-git builds
In reply to Bruce AshfieldThe ability to build non-git repositories was broken by two changes:
- The existence of an empty 'patches' directory created during the unpack phase. This dir was incorrectly identified as a valid meta directory and broke the build. By ensuring that it is removed before creating the empty repository, it will no longer be found instead of the real meta directory.
- The attempt to reset the git repository to a specific SRCREV when no SRCREV was provided. By checking for a SRCREV of 'INVALID', we avoid any processing and failed git operations.
--- meta/classes/kernel-yocto.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +3 -1
@@ -179,9 +179,11 @@ do_kernel_checkout() { bberror "S is not set to the linux source directory. Check " bbfatal "the recipe and set S to the proper extracted subdirectory" fi + rm -f .gitignore git init git add . git commit -q -m "baseline commit: creating repo for ${PN}-${PV}" + git clean -d -f fi # end debare
@@ -287,7 +289,7 @@ do_validate_branches() { if [ "${machine_srcrev}" = "AUTOINC" ]; then bbnote "SRCREV validation is not required for AUTOREV" elif [ "${machine_srcrev}" = "" ]; then - if [ "${SRCREV}" != "AUTOINC" ]; then + if [ "${SRCREV}" != "AUTOINC" ] && [ "${SRCREV}" != "INVALID" ]; then # SRCREV_machine_<MACHINE> was not set. This means that a custom recipe # that doesn't use the SRCREV_FORMAT "machine_meta" is being built. In # this case, we need to reset to the give SRCREV before heading to patching
Signature · 1 line
2.1.0
PATCH 7/7 kernel-yocto: make sure git tags get dereferenced properly in do_patch()
In reply to Bruce AshfieldFrom: Jeff Wang <[email protected]>
Commit 92c1ece6c347030d48995a36f4c67861356e59d3 causes the test in do_patch() in kernel-yocto.bbclass to fail if ${machine_srcrev} is an annotated tag. The check is meant to ensure that ${machine_srcrev} is an ancestor of HEAD, but if ${machine_srcrev} is a tag, then "$(git rev-parse --verify ${machine_srcrev})" evaluates to the SHA of the tag instead of what it's pointing to.
Replacing "$(git rev-parse --verify ${machine_srcrev})" with "$(git rev-parse --verify ${machine_srcrev}~0)" fixed the problem by finding the object pointed to by the tag, and not the tag itself. This also works for commit IDs that are not tags, hence is safe in a scenarios.
Jeff Wang <[email protected]>
--- meta/classes/kernel-yocto.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/kernel-yocto.bbclass +1 -1
@@ -129,7 +129,7 @@ do_patch() { # check to see if the specified SRCREV is reachable from the final branch. # if it wasn't something wrong has happened, and we should error. if [ "${machine_srcrev}" != "AUTOINC" ]; then - if ! [ "$(git rev-parse --verify ${machine_srcrev})" = "$(git merge-base ${machine_srcrev} HEAD)" ]; then + if ! [ "$(git rev-parse --verify ${machine_srcrev}~0)" = "$(git merge-base ${machine_srcrev} HEAD)" ]; then bberror "SRCREV ${machine_srcrev} was specified, but is not reachable" bbfatal "Check the BSP description for incorrect branch selection, or other errors." fi
Signature · 1 line
2.1.0