kernel/image/depmodwrapper: Fixups for depmod

With the rpm package backend enabled, running:

bitbake <image>
bitbake virtual/kernel -c clean
bitbake <image> -c rootfs -f

results in an image with incorrect kernel module dependency information. The problem is that the System.map and kernel-abiversion files are needed for depmod and after the recent kernel changes, these are no longer in sstate.

Its reasonable to require the kernel to unpack/build if you're about to build a module against it. It is not reasonable to require this just to build a rootfs.

Therefore stash the needed files specifically for depmod.

Also fix some STAGING_KERNEL_DIR references which were incorrect, found whilst sorting through his change.

This patch also makes the depmod files being missing a fatal error rather than something the system just ignores silently.

Signed-off-by Richard Purdie <[email protected]>

meta/classes/image.bbclass +1 -1

index 07e7f99..22b6970 100644
@@ -95,7 +95,7 @@ def rootfs_variables(d):
                   'SDK_OUTPUT','SDKPATHNATIVE','SDKTARGETSYSROOT','SDK_DIR','SDK_VENDOR','SDKIMAGE_INSTALL_COMPLEMENTARY','SDK_PACKAGE_ARCHS','SDK_OUTPUT','SDKTARGETSYSROOT','MULTILIBRE_ALLOW_REP',
                   'MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS','PACKAGE_ARCHS',
                   'PACKAGE_CLASSES','TARGET_VENDOR','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','BUILDNAME','USE_DEVFS',
-                 'STAGING_KERNEL_DIR','COMPRESSIONTYPES']
+                 'COMPRESSIONTYPES']
      variables.extend(command_variables(d))
      variables.extend(variable_depends(d))
      return " ".join(variables)

meta/classes/kernel-module-split.bbclass +1 -1

index 2d43b51..32b8085 100644
@@ -75,7 +75,7 @@ python split_kernel_module_packages () {
          if not os.path.exists(system_map_file):
              system_map_file = "%s/System.map-%s" % (staging_kernel_dir, kernelver)
              if not os.path.exists(system_map_file):
-                bb.fatal("System.map-%s does not exist in '%s/boot' nor STAGING_KERNEL_DIR '%s'" % (kernelver, dvar, staging_kernel_dir))
+                bb.fatal("System.map-%s does not exist in '%s/boot' nor STAGING_KERNEL_BUILDDIR '%s'" % (kernelver, dvar, staging_kernel_dir))
 
          cmd = "depmod -n -a -b %s -F %s %s" % (dvar, system_map_file, kernelver_stripped)
          f = os.popen(cmd, 'r')

meta/classes/kernel.bbclass +15 -3

index 805f799..2a16c71 100644
@@ -231,6 +231,11 @@ kernel_do_install() {
  	[ -e Module.symvers ] && install -m 0644 Module.symvers ${D}/boot/Module.symvers-${KERNEL_VERSION}
  	install -d ${D}${sysconfdir}/modules-load.d
  	install -d ${D}${sysconfdir}/modprobe.d
+
+	# Stash data for depmod
+	install -d ${D}${datadir}/kernel-depmod/
+	echo "${KERNEL_VERSION}" > ${D}${datadir}/kernel-depmod/kernel-abiversion
+	cp System.map ${D}${datadir}/kernel-depmod/System.map-${KERNEL_VERSION}
  }
  do_install[prefuncs] += "package_get_auto_pr"
 

@@ -280,14 +285,21 @@ do_shared_workdir () {
  	fi
  }
 
-# We have an empty sysroot_stage_all to keep the default routine from
-# package.bbclass from expanding the kernel source into the sysroot and
-# colliding with linux-firmware files
+# Only stage the files we need for depmod, not the modules/firmware
  sysroot_stage_all () {
+	sysroot_stage_dir ${D}${datadir}/kernel-depmod ${SYSROOT_DESTDIR}${datadir}/kernel-depmod
  }
 
  KERNEL_CONFIG_COMMAND ?= "oe_runmake_call -C ${S} O=${B} oldnoconfig || yes '' | oe_runmake -C ${S} O=${B} oldconfig"
 
+PACKAGE_PREPROCESS_FUNCS += "kernel_package_preprocess"
+
+kernel_package_preprocess () {
+    rm -rf ${PKGD}${datadir}/kernel-depmod
+    rmdir ${PKGD}${datadir}
+    rmdir ${PKGD}${exec_prefix}
+}
+
  kernel_do_configure() {
  	# fixes extra + in /lib/modules/2.6.37+
  	# $ scripts/setlocalversion . => +

meta/lib/oe/rootfs.py +8 -7

index f2891a7..72d32f7 100644
@@ -209,16 +209,17 @@ class Rootfs(object):
                                    'new', '-v'])
 
      def _generate_kernel_module_deps(self):
-        kernel_abi_ver_file = os.path.join(self.d.getVar('STAGING_KERNEL_BUILDDIR', True),
+        kernel_abi_ver_file = oe.path.join(self.d.getVar('STAGING_DIR_TARGET', True), self.d.getVar('datadir', True), "kernel-depmod",
                                             'kernel-abiversion')
-        if os.path.exists(kernel_abi_ver_file):
-            kernel_ver = open(kernel_abi_ver_file).read().strip(' \n')
-            modules_dir = os.path.join(self.image_rootfs, 'lib', 'modules', kernel_ver)
+        if not os.path.exists(kernel_abi_ver_file):
+            bb.fatal("No kernel-abiversion file found (%s), cannot run depmod, aborting" % kernel_abi_ver_file)
 
-            bb.utils.mkdirhier(modules_dir)
+        kernel_ver = open(kernel_abi_ver_file).read().strip(' \n')
+        modules_dir = os.path.join(self.image_rootfs, 'lib', 'modules', kernel_ver)
 
-            self._exec_shell_cmd(['depmodwrapper', '-a', '-b', self.image_rootfs,
-                                  kernel_ver])
+        bb.utils.mkdirhier(modules_dir)
+
+        self._exec_shell_cmd(['depmodwrapper', '-a', '-b', self.image_rootfs, kernel_ver])
 
      """
      Create devfs:

meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb +6 -6

index 7533809..83a382a 100644
@@ -19,21 +19,21 @@ if [ "\$1" != "-a" -o "\$2" != "-b" ]; then
      echo "Usage: depmodwrapper -a -b rootfs KERNEL_VERSION" >&2
      exit 1
  fi
-if [ ! -r ${STAGING_KERNEL_BUILDDIR}/kernel-abiversion ]; then
-    echo "Unable to read: ${STAGING_KERNEL_BUILDDIR}/kernel-abiversion" >&2
+if [ ! -r ${STAGING_DIR_TARGET}${datadir}/kernel-depmod/kernel-abiversion ]; then
+    echo "Unable to read: ${STAGING_DIR_TARGET}${datadir}/kernel-depmod/kernel-abiversion" >&2
  else
-    kernelabi=\$(cat ${STAGING_KERNEL_BUILDDIR}/kernel-abiversion)
+    kernelabi=\$(cat ${STAGING_DIR_TARGET}${datadir}/kernel-depmod/kernel-abiversion)
      if [ "\$kernelabi" != "\$4" ]; then
          echo "Error: Kernel version \$4 does not match kernel-abiversion (\$kernelabi)" >&2
          exit 1
      fi
  fi
 
-if [ ! -r ${STAGING_KERNEL_BUILDDIR}/System.map-\$4 ]; then
-    echo "Unable to read: ${STAGING_KERNEL_BUILDDIR}/System.map-\$4" >&2
+if [ ! -r ${STAGING_DIR_TARHET}${datadir}/kernel-depmod/System.map-\$4 ]; then
+    echo "Unable to read: ${STAGING_DIR_TARGET}${datadir}/kernel-depmod/System.map-\$4" >&2
      exec env depmod "\$1" "\$2" "\$3" "\$4"
  else
-    exec env depmod "\$1" "\$2" "\$3" -F "${STAGING_KERNEL_BUILDDIR}/System.map-\$4" "\$4"
+    exec env depmod "\$1" "\$2" "\$3" -F "${STAGING_DIR_TARGET}${datadir}/kernel-depmod/System.map-\$4" "\$4"
  fi
  EOF
  	chmod +x ${D}${bindir_crossscripts}/depmodwrapper

meta/recipes-kernel/systemtap/systemtap-uprobes_git.bb +1 -1

index 4d2bec4..758908b 100644
@@ -21,7 +21,7 @@ EXTRA_OEMAKE = ""
  # support.  Note that staprun expects it in the systemtap/runtime directory,
  # not in /lib/modules.
  do_compile() {
-	if grep -q "CONFIG_UTRACE=y" ${STAGING_KERNEL_DIR}/.config
+	if grep -q "CONFIG_UTRACE=y" ${STAGING_KERNEL_BUILDDIR}/.config
  	then
  		unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS CC LD CPP
  		oe_runmake CC="${KERNEL_CC}" LD="${KERNEL_LD}" \

Quoted 12 lines · Richard Purdie

With the rpm package backend enabled, running:

bitbake <image>
bitbake virtual/kernel -c clean
bitbake <image> -c rootfs -f

results in an image with incorrect kernel module dependency information. The problem is that the System.map and kernel-abiversion files are needed
for depmod and after the recent kernel changes, these are no longer in sstate.

Quoted 8 lines

Its reasonable to require the kernel to unpack/build if you're about to build a module against it. It is not reasonable to require this just to build a rootfs.

Therefore stash the needed files specifically for depmod.

Also fix some STAGING_KERNEL_DIR references which were incorrect, found whilst sorting through his change.

Now that I see the fix, the tweaks are clear (and obvious). I would have preferred a straight up module loading error in all cases .. since that would have showed in my workflows. The rootfs.py changes in particular would have taken me much longer to get right.

Quoted 1 line

Quoted 3 lines

This patch also makes the depmod files being missing a fatal error rather than
something the system just ignores silently.

Nice improvement.

Acked-by Bruce Ashfield <[email protected]>

Quoted 28 lines · Bruce Ashfield

Quoted 12 lines · Richard Purdie

With the rpm package backend enabled, running:

bitbake <image>
bitbake virtual/kernel -c clean
bitbake <image> -c rootfs -f

results in an image with incorrect kernel module dependency information. The problem is that the System.map and kernel-abiversion files are needed
for depmod and after the recent kernel changes, these are no longer in sstate.

Quoted 8 lines

Its reasonable to require the kernel to unpack/build if you're about to build a module against it. It is not reasonable to require this just to build a rootfs.

Therefore stash the needed files specifically for depmod.

Also fix some STAGING_KERNEL_DIR references which were incorrect, found whilst sorting through his change.

Now that I see the fix, the tweaks are clear (and obvious). I would have preferred a straight up module loading error in all cases .. since that would have showed in my workflows. The rootfs.py changes in particular would have taken me much longer to get right.

Quoted 1 line

Quoted 7 lines

Quoted 3 lines

This patch also makes the depmod files being missing a fatal error rather than
something the system just ignores silently.

Nice improvement.

Acked-by Bruce Ashfield <[email protected]>

Functionally look good. I noticed some whitespace issues (spaces in bash sections instead of tabs), but those may have been due to keeping a file internally consistent.

Otherwise,

Reviewed-by Darren Hart <[email protected]>