kernel: provide scripts in the sysroot

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Rather than further complicate the kernel's install rule, or its packaging, we can jump into the sysroot and restore the scripts by building them, hence making them available to packages that need them.

Signed-off-by Bruce Ashfield <[email protected]>

---
meta/classes/kernel.bbclass | 8 ++++++++

1 file changed, 8 insertions(+)

meta/classes/kernel.bbclass +0 -0

index 4acfb7e..d856e31 100644

meta/classes/kernel.bbclass +2 -0

+	  oe_runmake scripts 
+	)
 }
 
 kernel_do_configure() {

Signature · 1 line

1.8.1.2

Quoted 20 lines · Bruce Ashfield

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Rather than further complicate the kernel's install rule, or its packaging, we can jump into the sysroot and restore the scripts by building them, hence making them available to packages that need them.

Signed-off-by Bruce Ashfield <[email protected]>

---
meta/classes/kernel.bbclass | 8 ++++++++

1 file changed, 8 insertions(+)

meta/classes/kernel.bbclass +0 -0

index 4acfb7e..d856e31 100644
@@ -294,6 +294,14 @@ do_install[prefuncs] += "package_get_auto_pr"
 

meta/classes/kernel.bbclass do_install[prefuncs] += "package_get_auto_pr" · line 302

Quoted 8 lines

meta/classes/kernel.bbclass +6 -0

 sysroot_stage_all_append() {
 	sysroot_stage_dir ${D}${KERNEL_SRC_PATH} ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}
+
+	# Restore kernel scripts in the sysroot, since they had to be removed during
+	# packaging to avoid mismatched arch QA errors, but are required/useful when
+	# packages like out of tree kernel modules are built. Rather than playing QA
+	# games, it is easier to just do this work in the sysroot.
+	( cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH};

This was cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}/scripts in my patch but I will test out this one too. but this is ok since

ifeq ($(KBUILD_EXTMOD),)
# Additional helpers built in scripts/
# Carefully list dependencies so we do not try to build scripts twice

# in parallel
PHONY += scripts
scripts: scripts_basic include/config/auto.conf include/config/tristate.conf \
          asm-generic
         $(Q)$(MAKE) $(build)=$(@)

Quoted 8 lines

meta/classes/kernel.bbclass +2 -1

+	  oe_runmake scripts
+	)
 }
 
 kernel_do_configure() {
--

1.8.1.2

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Quoted 4 lines · Bruce Ashfield

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Its also to avoid mixing architectures when packaging the sstate for do_populate_sysroot. The sstate for that task is now native arch specific after this patch but its task hash is not. Even if we made it native specific, that means the kernel would rebuild entirely if you switch 32 bit to 64 bit native machine. We therefore cannot merge this patch as is.

Instead do something like:

SSTATEPOSTINSTFUNCS += "kernelheaders_sstate_postinst"

kernelheaders_sstate_postinst () {
	if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
	then
		( cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH};
		  oe_runmake scripts
		)
	fi
}

This will rerun the oe_runmake scripts each time the sstate package is installed. It slows down the use of sstate but should be correct whether the build machine is 32 or 64 bit.

Cheers,

Richard

Quoted 10 lines · Richard Purdie

Quoted 6 lines · Bruce Ashfield

When building against the sysroot, out of tree modules can require modpost
and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Quoted 6 lines

Its also to avoid mixing architectures when packaging the sstate for do_populate_sysroot. The sstate for that task is now native arch specific after this patch but its task hash is not. Even if we made it native specific, that means the kernel would rebuild entirely if you switch 32 bit to 64 bit native machine. We therefore cannot merge this patch as is.

Aha! I knew there was a lurking element I wouldn't know about. I nearly put "RFC" in the patch .. and clearly should have :)

Quoted 16 lines

Instead do something like:

SSTATEPOSTINSTFUNCS += "kernelheaders_sstate_postinst"

kernelheaders_sstate_postinst () {
        if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o
"${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
        then
                ( cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH};
                  oe_runmake scripts
                )
        fi
}

Quoted 3 lines

This will rerun the oe_runmake scripts each time the sstate package is installed. It slows down the use of sstate but should be correct whether the build machine is 32 or 64 bit.

I'll try what you have above and re-spin the patch.

Cheers,

Bruce

Quoted 7 lines

Cheers,

Richard

Quoted 4 lines · Bruce Ashfield

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Its also to avoid mixing architectures when packaging the sstate for do_populate_sysroot. The sstate for that task is now native arch specific after this patch but its task hash is not. Even if we made it native specific, that means the kernel would rebuild entirely if you switch 32 bit to 64 bit native machine. We therefore cannot merge this patch as is.

Instead do something like:

SSTATEPOSTINSTFUNCS += "kernelheaders_sstate_postinst"

kernelheaders_sstate_postinst () {
	if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
	then
		( cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH};
		  oe_runmake scripts
		)
	fi
}

This will rerun the oe_runmake scripts each time the sstate package is installed. It slows down the use of sstate but should be correct whether the build machine is 32 or 64 bit.

Cheers,

Richard

Quoted 27 lines · Richard Purdie

Quoted 4 lines · Bruce Ashfield

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Its also to avoid mixing architectures when packaging the sstate for do_populate_sysroot. The sstate for that task is now native arch specific after this patch but its task hash is not. Even if we made it native specific, that means the kernel would rebuild entirely if you switch 32 bit to 64 bit native machine. We therefore cannot merge this patch as is.

Instead do something like:

SSTATEPOSTINSTFUNCS += "kernelheaders_sstate_postinst"

kernelheaders_sstate_postinst () {
	if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
	then
		( cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH};
		  oe_runmake scripts
		)
	fi
}

Quoted 3 lines

This will rerun the oe_runmake scripts each time the sstate package is installed. It slows down the use of sstate but should be correct whether the build machine is 32 or 64 bit.

I poked with this a bit over the weekend, and never did get the right results.

I can make the scripts trigger, but since the sysroot population appears to already be done, but build scripts don't actually make it into the sysroot for use during module builds.

I tried a few variants of the below, all trying to get the actual tmp/sysroots/<machine> to have the scripts, with no luck. Dumping the available variables didn't get me anything really promising either.

I'll have another look in the morning, since I'm sure it is just me not understanding the ordering of things .. but any pointers would of course be appreciated.

Cheers,

Bruce

STATEPOSTINSTFUNCS += "kernelscripts_sstate_postinst"

kernelscripts_sstate_postinst () {
         if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o 
"${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]; then
                 sysroot_stage_dir ${D}${KERNEL_SRC_PATH} 
${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}
                 oe_runmake -C ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} scripts
	fi
} 

Quoted 52 lines · Bruce Ashfield

Quoted 27 lines · Richard Purdie

Quoted 4 lines · Bruce Ashfield

When building against the sysroot, out of tree modules can require modpost and other utilities normally found in the kernel's scripts directory. For the kernel source in the staging dir, these scripts have been removed to avoid mixing archiectures when packaging kernel-dev (among other things).

Its also to avoid mixing architectures when packaging the sstate for do_populate_sysroot. The sstate for that task is now native arch specific after this patch but its task hash is not. Even if we made it native specific, that means the kernel would rebuild entirely if you switch 32 bit to 64 bit native machine. We therefore cannot merge this patch as is.

Instead do something like:

SSTATEPOSTINSTFUNCS += "kernelheaders_sstate_postinst"

kernelheaders_sstate_postinst () {
	if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
	then
		( cd ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH};
		  oe_runmake scripts
		)
	fi
}

Quoted 3 lines

This will rerun the oe_runmake scripts each time the sstate package is installed. It slows down the use of sstate but should be correct whether the build machine is 32 or 64 bit.

I poked with this a bit over the weekend, and never did get the right results.

I can make the scripts trigger, but since the sysroot population appears to already be done, but build scripts don't actually make it into the sysroot for use during module builds.

I tried a few variants of the below, all trying to get the actual tmp/sysroots/<machine> to have the scripts, with no luck. Dumping the available variables didn't get me anything really promising either.

I'll have another look in the morning, since I'm sure it is just me not understanding the ordering of things .. but any pointers would of course be appreciated.

Cheers,

Bruce

STATEPOSTINSTFUNCS += "kernelscripts_sstate_postinst"

Quoted 9 lines

kernelscripts_sstate_postinst () {
          if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o
"${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]; then
                  sysroot_stage_dir ${D}${KERNEL_SRC_PATH}
${SYSROOT_DESTDIR}${KERNEL_SRC_PATH}
                  oe_runmake -C ${SYSROOT_DESTDIR}${KERNEL_SRC_PATH} scripts
	fi
}

Ah, I had it slightly wrong. Try:

kernelheaders_sstate_postinst () {
if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ] then
( cd ${KERNEL_SRC_PATH};
oe_runmake scripts
)
fi
}

since the files are actually installed at this point, therefore we operate on the final location.

The risk here is that when we uninstall the sstate package, we don't track the generated files. We can probably live with that for now.

What we really needs is a preinst but the one we have is really a preunpack :/.

Ultimately, I think we need to rename the preinst to preunpack and add a real preinst...

Cheers,

Richard

I had it slightly wrong. Try:

Quoted 10 lines

kernelheaders_sstate_postinst () {
if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ] then
( cd ${KERNEL_SRC_PATH};
oe_runmake scripts
)
fi
}

Quoted 2 lines

since the files are actually installed at this point, therefore we operate on the final location.

That's the kicker, I can't get the right variable to find the final location, KERNEL_SRC_PATH is set to /usr/src/kernel, so we can't operate on it directly. When things were runing in the sysroot_append, the kernel src was staged, and then operated on, then it makes it into the sysroot. Here, we could operate on the STAGING_KERNEL, which is in the sysroot, but stripped of the scripts. Perhaps that is the answer, but I need to confirm that scripts installed in that location would be available to the out of tree module builds that are looking for modpost and friends.

Quoted 14 lines · Bruce Ashfield

I had it slightly wrong. Try:

Quoted 10 lines

kernelheaders_sstate_postinst () {
   	if [ "${BB_CURRENTTASK}" = "populate_sysroot" -o "${BB_CURRENTTASK}" = "populate_sysroot_setscene" ]
   	then
   		( cd ${KERNEL_SRC_PATH};
   		  oe_runmake scripts
   		)
   	fi
   }

Quoted 2 lines

since the files are actually installed at this point, therefore we operate on the final location.

Quoted 9 lines

That's the kicker, I can't get the right variable to find the final location, KERNEL_SRC_PATH is set to /usr/src/kernel, so we can't operate on it directly. When things were runing in the sysroot_append, the kernel src was staged, and then operated on, then it makes it into the sysroot. Here, we could operate on the STAGING_KERNEL, which is in the sysroot, but stripped of the scripts. Perhaps that is the answer, but I need to confirm that scripts installed in that location would be available to the out of tree module builds that are looking for modpost and friends.

Guessing again (third time lucky?),

cd ${STAGING_DIR_KERNEL};

?

Cheers,

Richard