linux-headers: allow use of headers from kernel "package" selected
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
From: Karoly Kasza <[email protected]>
This is a re-work of http://patchwork.ozlabs.org/patch/346823/
In buildroot kernel headers used for building toolchain come from upstream kernel sources (even though user may specify version, but not origin).
This change makes it possible to use exactly the same sources for both headers during toolchain building and for kernel building itself.
Moreover if "kernel" is selected to be built by defaut its sources will be used for kernel headers on toolchain build.
That way user might be sure that ABI mismatch won't happen between toolchain and kernel.
---
Changes v1 -> v2:
* Make sure "Custom kernel headers series" is selected so BR2_TOOLCHAIN_HEADERS_AT_LEAST is properly set.
--- package/linux-headers/Config.in.host | 14 ++++++++++++-- package/linux-headers/linux-headers.mk | 7 +++++++ 2 files changed, 19 insertions(+), 2 deletions(-)
package/linux-headers/Config.in.host +12 -2
@@ -5,6 +5,7 @@ comment "Kernel Header Options" choice prompt "Kernel Headers" + default BR2_KERNEL_HEADERS_SAME_AS_KERNEL if BR2_LINUX_KERNEL default BR2_KERNEL_HEADERS_3_17 help Select the version of kernel header files you wish to use.
@@ -69,6 +70,13 @@ choice config BR2_KERNEL_HEADERS_VERSION bool "Manually specified Linux version" + + config BR2_KERNEL_HEADERS_SAME_AS_KERNEL + bool "Use Linux kernel specified for target" + depends on BR2_LINUX_KERNEL + help + Use the Linux kernel specified for the target in + the Kernel menu. endchoice config BR2_DEFAULT_KERNEL_VERSION
@@ -80,10 +88,12 @@ config BR2_DEFAULT_KERNEL_VERSION choice bool "Custom kernel headers series" - depends on BR2_KERNEL_HEADERS_VERSION + depends on BR2_KERNEL_HEADERS_VERSION || BR2_KERNEL_HEADERS_SAME_AS_KERNEL default BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_REALLY_OLD help - Set to the kernel headers series you manually set above. + Set to the kernel headers series you manually entered in + "linux version" field or matching version of specific kernel selected + for building in "kernel" section. This is used to hide/show some packages that have strict requirements on the version of kernel headers.
package/linux-headers/linux-headers.mk +7 -0
@@ -7,6 +7,12 @@ # This package is used to provide Linux kernel headers for the # internal toolchain backend. +ifeq ($(BR2_KERNEL_HEADERS_SAME_AS_KERNEL),y) +LINUX_HEADERS_VERSION = $(call qstrip,$(BR2_LINUX_KERNEL_VERSION)) +LINUX_HEADERS_SOURCE = $(LINUX_SOURCE) +LINUX_HEADERS_SITE = $(LINUX_SITE) +LINUX_HEADERS_SITE_METHOD = $(LINUX_SITE_METHOD) +else LINUX_HEADERS_VERSION = $(call qstrip,$(BR2_DEFAULT_KERNEL_HEADERS)) ifeq ($(findstring x2.6.,x$(LINUX_HEADERS_VERSION)),x2.6.) LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v2.6
@@ -14,6 +20,7 @@ else LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v3.x endif LINUX_HEADERS_SOURCE = linux-$(LINUX_HEADERS_VERSION).tar.xz +endif LINUX_HEADERS_INSTALL_STAGING = YES
Signature · 1 line
1.9.3
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
Hi,
I tested this patch, found a minor issue:
You can select a kernel, then select "Use Linux kernel specified for target" for toolchain headers, then go back to kernel and choose "Same as toolchain kernel headers" option. This makes BR2_KERNEL_HEADERS_SAME_AS_KERNEL=y BR2_LINUX_KERNEL_SAME_AS_HEADERS=y
selected together.
This produces this error:
---------------------------------------------------------------------------
Quoted 1 line
Quoted 1 line
Quoted 1 line
linux-headers Downloading
--2014-10-14 12:10:52--
https://www.kernel.org/pub/linux/kernel/v3.x/linux-.tar.xz
Resolving www.kernel.org (www.kernel.org)... 198.145.20.140, 149.20.4.69, 199.204.44.194, ...
Connecting to www.kernel.org (www.kernel.org)|198.145.20.140|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-10-14 12:10:54 ERROR 404: Not Found.
--2014-10-14 12:10:54-- http://sources.buildroot.net/linux-.tar.xz
Resolving sources.buildroot.net (sources.buildroot.net)... 176.9.16.109 Connecting to sources.buildroot.net (sources.buildroot.net)|176.9.16.109|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-10-14 12:10:54 ERROR 404: Not Found.
make: *** [/home/scythe/buildroot/output/build/linux-headers-/.stamp_downloaded] Error 1
---------------------------------------------------------------------------
This can be prevented by making a change in package/linux-headers/ Config.in.host:
...
+ config BR2_KERNEL_HEADERS_SAME_AS_KERNEL
+ bool "Use Linux kernel specified for target" + depends on BR2_LINUX_KERNEL + depends on !BR2_LINUX_KERNEL_SAME_AS_HEADERS
...
Regards,
Karoly
Quoted 104 lines · Alexey Brodkin
From: Karoly Kasza <[email protected]>
This is a re-work of http://patchwork.ozlabs.org/patch/346823/
In buildroot kernel headers used for building toolchain come from upstream kernel sources (even though user may specify version, but not origin).
This change makes it possible to use exactly the same sources for both headers during toolchain building and for kernel building itself.
Moreover if "kernel" is selected to be built by defaut its sources will be used for kernel headers on toolchain build.
That way user might be sure that ABI mismatch won't happen between toolchain
and kernel.Signed-off-by Alexey Brodkin <[email protected]>Signed-off-by Karoly Kasza <[email protected]>Cc Peter Korsgaard <[email protected]>---
Changes v1 -> v2:
* Make sure "Custom kernel headers series" is selected soBR2_TOOLCHAIN_HEADERS_AT_LEASTis properly set.--- package/linux-headers/Config.in.host | 14 ++++++++++++-- package/linux-headers/linux-headers.mk | 7 +++++++ 2 files changed, 19 insertions(+), 2 deletions(-)package/linux-headers/Config.in.host +9
-1@@ -5,6 +5,7 @@ comment "Kernel Header Options" choice prompt "Kernel Headers" + default BR2_KERNEL_HEADERS_SAME_AS_KERNEL if BR2_LINUX_KERNEL default BR2_KERNEL_HEADERS_3_17 help Select the version of kernel header files you wish to use.@@ -69,6 +70,13 @@ choice config BR2_KERNEL_HEADERS_VERSION bool "Manually specified Linux version" + + config BR2_KERNEL_HEADERS_SAME_AS_KERNEL + bool "Use Linux kernel specified for target" + depends on BR2_LINUX_KERNEL + help + Use the Linux kernel specified for the target in + the Kernel menu. endchoice config BR2_DEFAULT_KERNEL_VERSION@@ -80,10 +88,12 @@ config BR2_DEFAULT_KERNEL_VERSION choice bool "Custom kernel headers series" - depends on BR2_KERNEL_HEADERS_VERSION + depends on BR2_KERNEL_HEADERS_VERSION ||
BR2_KERNEL_HEADERS_SAME_AS_KERNEL
defaultBR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_REALLY_OLDhelppackage/linux-headers/Config.in.host +2
-1- Set to the kernel headers series you manually set above. + Set to the kernel headers series you manually entered in + "linux version" field or matching version of specific kernelselected
package/linux-headers/Config.in.host +1
-0+ for building in "kernel" section. This is used to hide/show some packages that have strict requirements on the version of kernel headers.package/linux-headers/linux-headers.mk +7
-1@@ -7,6 +7,12 @@ # This package is used to provide Linux kernel headers for the # internal toolchain backend. +ifeq ($(BR2_KERNEL_HEADERS_SAME_AS_KERNEL),y) +LINUX_HEADERS_VERSION = $(call qstrip,$(BR2_LINUX_KERNEL_VERSION)) +LINUX_HEADERS_SOURCE = $(LINUX_SOURCE) +LINUX_HEADERS_SITE = $(LINUX_SITE) +LINUX_HEADERS_SITE_METHOD = $(LINUX_SITE_METHOD) +else LINUX_HEADERS_VERSION = $(call qstrip,$(BR2_DEFAULT_KERNEL_HEADERS)) ifeq ($(findstring x2.6.,x$(LINUX_HEADERS_VERSION)),x2.6.) LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v2.6@@ -14,6 +20,7 @@ else LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v3.x endif LINUX_HEADERS_SOURCE = linux-$(LINUX_HEADERS_VERSION).tar.xz +endif LINUX_HEADERS_INSTALL_STAGING = YES --1.9.3
Signature · 5 lines
?dv, KK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20141014/e5e55908/attachment.html>
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
Hi Karoly,
Quoted 32 lines · K?roly Kasza
Hi,
I tested this patch, found a minor issue:
You can select a kernel, then select "Use Linux kernel specified for target" for toolchain headers, then go back to kernel and choose "Same as toolchain kernel headers" option. This makesBR2_KERNEL_HEADERS_SAME_AS_KERNEL=y
BR2_LINUX_KERNEL_SAME_AS_HEADERS=y
selected together.
This produces this error:---------------------------------------------------------------------------
Quoted 1 line
Quoted 1 line
Quoted 1 line
linux-headers Downloading
--2014-10-14 12:10:52--
https://www.kernel.org/pub/linux/kernel/v3.x/linux-.tar.xz
Resolving www.kernel.org (www.kernel.org)... 198.145.20.140, 149.20.4.69, 199.204.44.194, ...
Connecting to www.kernel.org (www.kernel.org)|198.145.20.140|:443... connected.
HTTP request sent, awaiting response... 404 Not Found2014-10-14 12:10:54 ERROR 404: Not Found.--2014-10-14 12:10:54-- http://sources.buildroot.net/linux-.tar.xz
Resolving sources.buildroot.net (sources.buildroot.net)... 176.9.16.109
Connecting to sources.buildroot.net (sources.buildroot.net)| 176.9.16.109|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
Quoted 7 lines
2014-10-14 12:10:54 ERROR 404: Not Found.make: *** [
/home/scythe/buildroot/output/build/linux-headers-/.stamp_downloaded] Error 1
---------------------------------------------------------------------------
Well I sort of expected those kind of issues with cross-dependent features.
package/linux-headers/Config.in.host choice
Quoted 9 lines
This can be prevented by making a change
in package/linux-headers/Config.in.host:
...package/linux-headers/Config.in.host +4
-0+ config BR2_KERNEL_HEADERS_SAME_AS_KERNEL + bool "Use Linux kernel specified for target" + depends on BR2_LINUX_KERNEL + depends on !BR2_LINUX_KERNEL_SAME_AS_HEADERS
Thanks for this, but unfortunately this is not the only problem. Much more significant one is how to get kernel sources for "linux-headers" package from the same location as "linux" package will get them.
We cannot rely on internal variables of "linux" package like LINUX_xxx because they could be resolved later than we need them.
So we need to do the same one calculations as we do in "linux.mk" see:linux/linux.mk: # Compute LINUX_SOURCE and LINUX_SITE from the configuration
In the best case it's dummy code duplication which inevitably will bite us in the future. Still I didn't find any better solution. Any input is much appreciated.
-Alexey
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
Hi Alexey,
Much more significant one is how to get kernel sources for
Quoted 6 lines
"linux-headers" package from the same location as "linux" package will get them.
We cannot rely on internal variables of "linux" package like
LINUX_xxxbecause they could be resolved later than we need them.
Could you point me some examples, in which cases can these be computed too late?
Quoted 4 lines
So we need to do the same one calculations as we do in "linux.mk" see:
linux/linux.mk: # ComputeLINUX_SOURCEandLINUX_SITEfrom the configuration
Maybe add an extra target for computing the variables somewhere, and make both
linux and linux-headers depend on it?
Quoted 4 lines
In the best case it's dummy code duplication which inevitably will bite us in the future. Still I didn't find any better solution. Any input is much appreciated.
I agree, code duplication is at least not future-proof :)
Best regards,
Karoly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20141025/1092b844/attachment.html>
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
Hi Karoly,
Quoted 2 lines · K?roly Kasza
Could you point me some examples, in which cases can these be computed too late?
The point is we cannot know for sure and moreover guarantee when internal variables are calculated.
Remember we're dealing with "make" utility that does all calculations at once on start. So it has nothing to do which real or virtual package (like linux kernel) gets built first.
The only reliable values are BR2_xxx which are set in ".config" file on exit from Buildroot configuration.
But we don't to put each and every tiny thing in ".config" that's why we decided to do another round of calculation of kernel source path in "linux-headers" based on values already available in ".config".
Quoted 8 lines
So we need to do the same one calculations as we do in "linux.mk" see:
linux/linux.mk: # ComputeLINUX_SOURCEandLINUX_SITEfrom the configuration
Maybe add an extra target for computing the variables somewhere, and make both
linux and linux-headers depend on it?
Well, frankly I don't quite understand how to do this kind of 2 stage variables resolution with "make".
Probably another approach could be to eliminate "linux-headers" package completely and modify "linux" virtual package so it could be only downloaded, extracted and headers-installed if user doesn't want to build kernel itself. I.e. move all functionality of "linux-headers" to "linux".
But then we'll need to migrate kernel version (well headers version) functionality as well so we still have properly resolved dependencies for packages which require headers newer than XXX.
In other words scope of work here looks like pretty significant (if done properly) but benefits are not that obvious.
Because IMHO the only reason you may want to use the same sources/headers for tools and kernel if there is a ABI breakage which for most of modern architectures not feasible.
Existing architectures seem to not suffer from it - their ABIs are stable for quite a long time and new architectures are forced to use UAPI so most of things will be unified.
Still if you do need this functionality nobody may stop you from trying and I'll happily help you with details, reviews, tests etc.
-Alexey
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
Hi Alexey
Quoted 6 lines
make both
Quoted 1 line
linux and linux-headers depend on it?
Well, frankly I don't quite understand how to do this kind of 2 stage variables resolution with "make".
I meant adding a dummy target, like linux-paths, that calculates the variables, and both linux and linux-headers
as targets would depend on it.
I'm not sure if this is possible without the much ugly hacks, so I just evaluating right now.
My other idea is to use a Makefile function for calculation and call the function from both places.
Quoted 6 lines
Probably another approach could be to eliminate "linux-headers" package completely and modify "linux" virtual package so it could be only downloaded, extracted and headers-installed if user doesn't want to build kernel itself. I.e. move all functionality of "linux-headers" to "linux".
Quoted 7 lines
But then we'll need to migrate kernel version (well headers version) functionality as well so we still have properly resolved dependencies for packages which require headers newer than XXX.
In other words scope of work here looks like pretty significant (if done properly) but benefits are not that obvious.
Probably you're right. Maybe this will have use in the future, but only if using a Buildroot toolchain
built with the system.
Quoted 9 lines
Because IMHO the only reason you may want to use the same sources/headers for tools and kernel if there is a ABI breakage which for most of modern architectures not feasible.
Existing architectures seem to not suffer from it - their ABIs are stable for quite a long time and new architectures are forced to use UAPI so most of things will be unified.
True, I just think it's a much cleaner and more elegant approach to use the same sources for headers and kernel
if using a custom kernel (and not just adding patches at compile time with BR2_LINUX_KERNEL_PATCH to vanilla)
and own toolchain.
Quoted 3 lines
Still if you do need this functionality nobody may stop you from trying and I'll happily help you with details, reviews, tests etc.
Thank you!
Best regards,
Karoly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20141028/a85b8773/attachment.html>
PATCH v2 1/1 linux-headers: allow use of headers from kernel "package" selected
In reply to Károly KaszaOn 2014-10-14 13:16 +0200, K?roly Kasza spake thusly:
Quoted 37 lines
Hi,
I tested this patch, found a minor issue:
You can select a kernel, then select "Use Linux kernel specified for target" for toolchain headers, then go back to kernel and choose "Same as toolchain kernel headers" option. This makes BR2_KERNEL_HEADERS_SAME_AS_KERNEL=y BR2_LINUX_KERNEL_SAME_AS_HEADERS=y
selected together.
This produces this error:---------------------------------------------------------------------------
Quoted 1 line
Quoted 1 line
Quoted 1 line
linux-headers Downloading
--2014-10-14 12:10:52--
https://www.kernel.org/pub/linux/kernel/v3.x/linux-.tar.xz
Resolving www.kernel.org (www.kernel.org)... 198.145.20.140, 149.20.4.69, 199.204.44.194, ...
Connecting to www.kernel.org (www.kernel.org)|198.145.20.140|:443... connected.
HTTP request sent, awaiting response... 404 Not Found2014-10-14 12:10:54 ERROR 404: Not Found.--2014-10-14 12:10:54-- http://sources.buildroot.net/linux-.tar.xz
Resolving sources.buildroot.net (sources.buildroot.net)... 176.9.16.109 Connecting to sources.buildroot.net (sources.buildroot.net)|176.9.16.109|:80... connected.
HTTP request sent, awaiting response... 404 Not Found2014-10-14 12:10:54 ERROR 404: Not Found.make: *** [
/home/scythe/buildroot/output/build/linux-headers-/.stamp_downloaded] Error 1
---------------------------------------------------------------------------This can be prevented by making a change in package/linux-headers/ Config.in.host:
...
package/linux-headers/Config.in.host choice
Quoted 4 lines
package/linux-headers/Config.in.host +4
-0+ config BR2_KERNEL_HEADERS_SAME_AS_KERNEL + bool "Use Linux kernel specified for target" + depends on BR2_LINUX_KERNEL + depends on !BR2_LINUX_KERNEL_SAME_AS_HEADERS
Actually, I'd prefer the other way around:
config BR2_LINUX_KERNEL_SAME_AS_HEADERS
depends on !BR2_KERNEL_HEADERS_SAME_AS_KERNEL
Because it makes more sense to have the headers be the same as the running kernel, rather than have the kernel be the same as the one used for the headers.
(logically, it is the same, but the first one means that the headers derive from the running kernel, which is IMHO much more appropriate.)
Regards,
Yann E. MORIN.
Quoted 125 lines
...
Regards,
KarolyQuoted 104 lines · Alexey Brodkin
From: Karoly Kasza <[email protected]>
This is a re-work of http://patchwork.ozlabs.org/patch/346823/
In buildroot kernel headers used for building toolchain come from upstream kernel sources (even though user may specify version, but not origin).
This change makes it possible to use exactly the same sources for both headers during toolchain building and for kernel building itself.
Moreover if "kernel" is selected to be built by defaut its sources will be used for kernel headers on toolchain build.
That way user might be sure that ABI mismatch won't happen between toolchain
and kernel.Signed-off-by Alexey Brodkin <[email protected]>Signed-off-by Karoly Kasza <[email protected]>Cc Peter Korsgaard <[email protected]>---
Changes v1 -> v2:
* Make sure "Custom kernel headers series" is selected soBR2_TOOLCHAIN_HEADERS_AT_LEASTis properly set.--- package/linux-headers/Config.in.host | 14 ++++++++++++-- package/linux-headers/linux-headers.mk | 7 +++++++ 2 files changed, 19 insertions(+), 2 deletions(-)package/linux-headers/Config.in.host +9
-1@@ -5,6 +5,7 @@ comment "Kernel Header Options" choice prompt "Kernel Headers" + default BR2_KERNEL_HEADERS_SAME_AS_KERNEL if BR2_LINUX_KERNEL default BR2_KERNEL_HEADERS_3_17 help Select the version of kernel header files you wish to use.@@ -69,6 +70,13 @@ choice config BR2_KERNEL_HEADERS_VERSION bool "Manually specified Linux version" + + config BR2_KERNEL_HEADERS_SAME_AS_KERNEL + bool "Use Linux kernel specified for target" + depends on BR2_LINUX_KERNEL + help + Use the Linux kernel specified for the target in + the Kernel menu. endchoice config BR2_DEFAULT_KERNEL_VERSION@@ -80,10 +88,12 @@ config BR2_DEFAULT_KERNEL_VERSION choice bool "Custom kernel headers series" - depends on BR2_KERNEL_HEADERS_VERSION + depends on BR2_KERNEL_HEADERS_VERSION ||
BR2_KERNEL_HEADERS_SAME_AS_KERNEL
defaultBR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_REALLY_OLDhelppackage/linux-headers/Config.in.host +2
-1- Set to the kernel headers series you manually set above. + Set to the kernel headers series you manually entered in + "linux version" field or matching version of specific kernelselected
package/linux-headers/Config.in.host +1
-0+ for building in "kernel" section. This is used to hide/show some packages that have strict requirements on the version of kernel headers.package/linux-headers/linux-headers.mk +7
-1@@ -7,6 +7,12 @@ # This package is used to provide Linux kernel headers for the # internal toolchain backend. +ifeq ($(BR2_KERNEL_HEADERS_SAME_AS_KERNEL),y) +LINUX_HEADERS_VERSION = $(call qstrip,$(BR2_LINUX_KERNEL_VERSION)) +LINUX_HEADERS_SOURCE = $(LINUX_SOURCE) +LINUX_HEADERS_SITE = $(LINUX_SITE) +LINUX_HEADERS_SITE_METHOD = $(LINUX_SITE_METHOD) +else LINUX_HEADERS_VERSION = $(call qstrip,$(BR2_DEFAULT_KERNEL_HEADERS)) ifeq ($(findstring x2.6.,x$(LINUX_HEADERS_VERSION)),x2.6.) LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v2.6@@ -14,6 +20,7 @@ else LINUX_HEADERS_SITE = $(BR2_KERNEL_MIRROR)/linux/kernel/v3.x endif LINUX_HEADERS_SOURCE = linux-$(LINUX_HEADERS_VERSION).tar.xz +endif LINUX_HEADERS_INSTALL_STAGING = YES --1.9.3
Signature · 7 lines
?dv, KK _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'