kernelsrc.bbclass: Move virtual/kernel:do_shared_workdir to do_configure
cover kernelsrc.bbclass: Move virtual/kernel:do_shared_workdir to do_configure
From: Robert Yang <[email protected]>
The following changes since commit 20a64bc696033b4d67294058b6c4a2e8f29f3aaa:
gsettings-desktop-schemas: update 46.1 -> 47.1 (2024-09-20 15:56:27 +0100)
are available in the Git repository at:
https://github.com/robertlinux/yocto rbt/kernel
https://github.com/robertlinux/yocto/tree/rbt/kernel
Robert Yang (1):
kernelsrc.bbclass: Move virtual/kernel:do_shared_workdir to
do_configure
meta/classes-recipe/kernelsrc.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Signature · 1 line
2.25.1
PATCH 1/1 kernelsrc.bbclass: Move virtual/kernel:do_shared_workdir to do_configure
From: Robert Yang <[email protected]>
The do_shared_workdir populates build artifacts to work-shared/${MACHINE}/kernel-build-artifacts, which isn't useful for do_patch, make do_patch depend on it will make a lot of recipes compiling which makes world's do_patch very slow, e.g.:
Download sources locally:
$ bitbake world --runall=fetch
* Before the patch:
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 23m31.739s user 0m25.086s sys 0m4.630s
* Now
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 12m25.650s user 0m35.641s sys 0m5.699s
--- meta/classes-recipe/kernelsrc.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes-recipe/kernelsrc.bbclass +1 -1
@@ -7,8 +7,8 @@ S = "${STAGING_KERNEL_DIR}" deltask do_fetch deltask do_unpack -do_patch[depends] += "virtual/kernel:do_shared_workdir" do_patch[noexec] = "1" +do_configure[depends] += "virtual/kernel:do_shared_workdir" do_package[depends] += "virtual/kernel:do_populate_sysroot" KERNEL_VERSION = "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}" LOCAL_VERSION = "${@get_kernellocalversion_file("${STAGING_KERNEL_BUILDDIR}")}"
Signature · 1 line
2.25.1
Quoted 17 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
do_shared_workdirpopulates build artifacts to work-shared/${MACHINE}/kernel-build-artifacts, which isn't useful fordo_patch, makedo_patchdepend on it will make a lot of recipes compiling which makes world'sdo_patchvery slow, e.g.:Download sources locally:
$ bitbake world --runall=fetch* Before the patch:
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 23m31.739s user 0m25.086s sys 0m4.630s
Quoted 5 lines
* Now
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 12m25.650s user 0m35.641s sys 0m5.699s
The motivation for why this is a priority should be included in the commit message. It helps us decide if any issues this may trigger are worth the risk.
I assume you are doing some sort of source archiving, or licensing and all you care about is the patched source for this, and not any potentially generated files from the build(s) ?
See below for why I ask.
Quoted 11 lines
Signed-off-by Robert Yang <[email protected]>--- meta/classes-recipe/kernelsrc.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes-recipe/kernelsrc.bbclass +0
-0
meta/classes-recipe/kernelsrc.bbclass line 11
Quoted 6 lines
meta/classes-recipe/kernelsrc.bbclass +1
-1@@ -7,8 +7,8 @@ S = "${STAGING_KERNEL_DIR}" deltask do_fetch deltask do_unpack -do_patch[depends] += "virtual/kernel:do_shared_workdir" do_patch[noexec] = "1" +do_configure[depends] += "virtual/kernel:do_shared_workdir"
From what I can see (bitbake expansion and dependency experts can feel free to correct the below):
There's already an issue with the KERNEL_VERSION / LOCAL_VERSION variables, in that they depend on STAGING_KERNEL_BUILDDIR, which
is populated from the do_shared_workdir. We are moving that population later / to a different task with this patch, so those variables are now going to be defined at a different point in the build. There's potential risk in that.
Looking at the routines get_kernel* that the variables are using, they gracefully return None if the file doesn't exist, so the question is .. would an impacted user of the variables notice a potential issue ?
One wonders if they shouldn't just propagate the exception to the caller so it could be dealt with, and harder to ignore.
So while it does speed up do_patch, there is a behaviour change in when the variables are going to be valid/defined.
Bruce
Quoted 15 lines
meta/classes-recipe/kernelsrc.bbclass +0
-1do_package[depends] += "virtual/kernel:do_populate_sysroot" KERNEL_VERSION = "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}" LOCAL_VERSION = "${@get_kernellocalversion_file("${STAGING_KERNEL_BUILDDIR}")}" --2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204794): https://lists.openembedded.org/g/openembedded-core/message/204794
Mute This Topic: https://lists.openembedded.org/mt/108603063/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Hi Bruce,
Sorry for the late reply, I just came back from holiday,
Quoted 25 lines · Bruce Ashfield
Quoted 17 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
do_shared_workdirpopulates build artifacts to work-shared/${MACHINE}/kernel-build-artifacts, which isn't useful fordo_patch, makedo_patchdepend on it will make a lot of recipes compiling which makes world'sdo_patchvery slow, e.g.:Download sources locally:
$ bitbake world --runall=fetch* Before the patch:
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 23m31.739s user 0m25.086s sys 0m4.630sQuoted 5 lines
* Now
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 12m25.650s user 0m35.641s sys 0m5.699s
Quoted 8 lines
The motivation for why this is a priority should be included in the commit message. It helps us decide if any issues this may trigger are worth the risk.
I assume you are doing some sort of source archiving, or licensing and all you care about is the patched source for this, and not any potentially generated files from the build(s) ?
Yes, I'm trying to collect the patched sources for ip scanning, and there are a lot of do_compile tasks are running when run --runall=patch which looks strange.
Quoted 30 lines
See below for why I ask.
Quoted 18 lines
Signed-off-by Robert Yang <[email protected]>--- meta/classes-recipe/kernelsrc.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes-recipe/kernelsrc.bbclass +1
-1@@ -7,8 +7,8 @@ S = "${STAGING_KERNEL_DIR}" deltask do_fetch deltask do_unpack -do_patch[depends] += "virtual/kernel:do_shared_workdir" do_patch[noexec] = "1" +do_configure[depends] += "virtual/kernel:do_shared_workdir"Quoted 1 line
From what I can see (bitbake expansion and dependency experts
can feel free to correct the below):
There's already an issue with the
KERNEL_VERSION/LOCAL_VERSIONvariables, in that they depend onSTAGING_KERNEL_BUILDDIR, which
is populated from thedo_shared_workdir. We are moving that population later / to a different task with this patch, so those variables are now going to be defined at a different point in the build. There's potential risk in that.
Quoted 3 lines
Looking at the routines get_kernel* that the variables are using, they gracefully return None if the file doesn't exist, so the question is .. would an impacted user of the variables notice a potential issue ?
Maybe we should make get_kernel_* report errors rather than None?
// Robert
Quoted 26 lines
One wonders if they shouldn't just propagate the exception to the caller so it could be dealt with, and harder to ignore.
So while it does speed up
do_patch, there is a behaviour change in when the variables are going to be valid/defined.Bruce
Quoted 15 lines
do_package[depends] += "virtual/kernel:do_populate_sysroot"
KERNEL_VERSION= "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}"LOCAL_VERSION= "${@get_kernellocalversion_file("${STAGING_KERNEL_BUILDDIR}")}"
--
2.25.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204794): https://lists.openembedded.org/g/openembedded-core/message/204794
Mute This Topic: https://lists.openembedded.org/mt/108603063/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 73 lines · Robert Yang
Hi Bruce,
Sorry for the late reply, I just came back from holiday,
Quoted 25 lines · Bruce Ashfield
Quoted 17 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
do_shared_workdirpopulates build artifacts to work-shared/${MACHINE}/kernel-build-artifacts, which isn't useful fordo_patch, makedo_patchdepend on it will make a lot of recipes compiling which makes world'sdo_patchvery slow, e.g.:Download sources locally:
$ bitbake world --runall=fetch* Before the patch:
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 23m31.739s user 0m25.086s sys 0m4.630sQuoted 5 lines
* Now
$ rm -fr sstate-cache/ tmp/; time bitbake world --runall=patch real 12m25.650s user 0m35.641s sys 0m5.699sQuoted 8 lines
The motivation for why this is a priority should be included in the commit message. It helps us decide if any issues this may trigger are worth the risk.
I assume you are doing some sort of source archiving, or licensing and all you care about is the patched source for this, and not any potentially generated files from the build(s) ?
Yes, I'm trying to collect the patched sources for ip scanning, and there are a lot of
do_compiletasks are running when run --runall=patch which looks strange.Quoted 30 lines
See below for why I ask.
Quoted 18 lines
Signed-off-by Robert Yang <[email protected]>--- meta/classes-recipe/kernelsrc.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes-recipe/kernelsrc.bbclass +1
-1@@ -7,8 +7,8 @@ S = "${STAGING_KERNEL_DIR}" deltask do_fetch deltask do_unpack -do_patch[depends] += "virtual/kernel:do_shared_workdir" do_patch[noexec] = "1" +do_configure[depends] += "virtual/kernel:do_shared_workdir"Quoted 1 line
From what I can see (bitbake expansion and dependency experts
can feel free to correct the below):
There's already an issue with the
KERNEL_VERSION/LOCAL_VERSIONvariables, in that they depend onSTAGING_KERNEL_BUILDDIR, which
is populated from thedo_shared_workdir. We are moving that population later / to a different task with this patch, so those variables are now going to be defined at a different point in the build. There's potential risk in that.
Quoted 5 lines
Quoted 3 lines
Looking at the routines get_kernel* that the variables are using, they gracefully return None if the file doesn't exist, so the question is .. would an impacted user of the variables notice a potential issue ?
Maybe we should make get_kernel_* report errors rather than None?
That was the idea that Ross and I had when I was looking through the series.
It would definitely let us know if this is an issue, and flag it in the future if it becomes an issue.
Bruce
Quoted 29 lines
// Robert
Quoted 26 lines
One wonders if they shouldn't just propagate the exception to the caller so it could be dealt with, and harder to ignore.
So while it does speed up
do_patch, there is a behaviour change in when the variables are going to be valid/defined.Bruce
Quoted 15 lines
do_package[depends] += "virtual/kernel:do_populate_sysroot"
KERNEL_VERSION= "${@get_kernelversion_file("${STAGING_KERNEL_BUILDDIR}")}"LOCAL_VERSION= "${@get_kernellocalversion_file("${STAGING_KERNEL_BUILDDIR}")}"
--
2.25.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204794): https://lists.openembedded.org/g/openembedded-core/message/204794
Mute This Topic: https://lists.openembedded.org/mt/108603063/1050810
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II