Need suggestions: in case where linux-lib-headers and the kernel versions differ
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:
1) Machine is supported on 5.4 kernel
*Kernel Recipe:* linux-yocto_5.4.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb There is no conflict in this case as libc-headers and linux-yocto are on same version.
2) Machine is supported on a higher kernel (let's say 5.10)
*Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.
3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully. What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3 - videodev2.h: This header is enhanced in 5.4
- fsverity.h : this header is not present in 4.19
--
Regards,
Sourabh
Hi Sourabh,
Quoted 15 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
*Kernel Recipe:* linux-yocto_5.4.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb There is no conflict in this case as libc-headers and linux-yocto are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
*Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.
Quoted 6 lines
3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully. What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
The issue is that linux-libc-headers is not machine-specific, only distro-specific, so you'd need this PREFERRED_VERSION_linux-libc-headers specified in a distro (or local.conf for that matter, but... NO, please NO :) ).
You could also technically provide your own linux-libc-headers in a layer of a higher priority and then PREFERRED_VERSION_linux-libc-headers won't matter as the highest prio layer's recipe will take precedence. But that is IMO bad practice as it means your distro has a different linux-libc-headers depending on whether a layer is included or not.
I don't have a solution personally, just wanted to share why the last suggestion does not seem correct to me.
Cheers,
Quentin
Quoted 16 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
Kernel Recipe: linux-yocto_5.4.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb There is no conflict in this case as libc-headers and linux-yocto are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
Kernel Recipe: Let's assume provided by a BSP layer, linux_5.10.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.
Quoted 6 lines
3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully. What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
This is not something that we've really seen happen. When the libc-headers are property used, they'll check/probe for functionality and use what is available.
If you have drivers/applications going directly at the libc-headers, versus the toolchain use. That is a different question. In this case, you'd be better of dealing with these on a per-application basis, and pointing them at alternate headers.
You can dream up all sorts of issues, and most of them don't happen in practice. I've run hundreds of kernels that are older than the libc-headers, and have never run into an issue with standard, non-machine specific applications.
Just look at master of oe-core. We continue to support 5.4, after I've bumped the headers to 5.10.
Cheers,
Bruce
Quoted 9 lines
I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
-videodev2.h: This header is enhanced in 5.4
-fsverity.h: this header is not present in 4.19--
Regards,
Sourabh
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Quoted 36 lines · Bruce Ashfield
[Please note: This e-mail is from an EXTERNAL e-mail address]
Quoted 16 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
Kernel Recipe: linux-yocto_5.4.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb There is no conflict in this case as libc-headers and linux-yocto are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
Kernel Recipe: Let's assume provided by a BSP layer, linux_5.10.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.Quoted 6 lines
3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully. What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"This is not something that we've really seen happen. When the libc-headers are property used, they'll check/probe for functionality and use what is available.
If you have
drivers/applicationsgoing directly at the libc-headers, versus the toolchain use. That is a different question. In this case, you'd be better of dealing with these on a per-application basis, and pointing them at alternate headers.
Quoted 8 lines
You can dream up all sorts of issues, and most of them don't happen in practice. I've run hundreds of kernels that are older than the libc-headers, and have never run into an issue with standard, non-machine specific applications.
Just look at master of oe-core. We continue to support 5.4, after I've bumped the headers to 5.10.
maybe unrelated and anecdotal:
I looked at a windriver bug related to a test in libnl called test-genl
The failure only occurred in 5.4.X kernel.
and I was able to narrow it down to the 5.10 header update oe-core causing the failure.
(if you are interested in details, talk to randy...)
Quoted 24 lines
Cheers,
Bruce
Quoted 9 lines
I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
-videodev2.h: This header is enhanced in 5.4
-fsverity.h: this header is not present in 4.19--
Regards,
Sourabh--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Quoted 46 lines · Yi Fan Yu
Quoted 36 lines · Bruce Ashfield
[Please note: This e-mail is from an EXTERNAL e-mail address]
Quoted 16 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
Kernel Recipe: linux-yocto_5.4.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb There is no conflict in this case as libc-headers and linux-yocto are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
Kernel Recipe: Let's assume provided by a BSP layer, linux_5.10.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as 5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.Quoted 6 lines
3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully. What's the recommendation here? Should the BSP provider port linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"This is not something that we've really seen happen. When the libc-headers are property used, they'll check/probe for functionality and use what is available.
If you have
drivers/applicationsgoing directly at the libc-headers, versus the toolchain use. That is a different question. In this case, you'd be better of dealing with these on a per-application basis, and pointing them at alternate headers.Quoted 8 lines
You can dream up all sorts of issues, and most of them don't happen in practice. I've run hundreds of kernels that are older than the libc-headers, and have never run into an issue with standard, non-machine specific applications.
Just look at master of oe-core. We continue to support 5.4, after I've bumped the headers to 5.10.
Quoted 8 lines
maybe unrelated and anecdotal:
I looked at a windriver bug related to a test in libnl called
test-genlThe failure only occurred in 5.4.X kernel.
and I was able to narrow it down to the 5.10 header update oe-core causing the failure.
libnl is known for this. It's basically the ioctl of the networking world.
So that's consistent with what I said, it is rare, and is usually in tightly coupled applications. When that happens, we almost always just carry a header or two along with the application itself.
Bruce
Quoted 26 lines
(if you are interested in details, talk to randy...)
Quoted 24 lines
Cheers,
Bruce
Quoted 9 lines
I suspect this as UAPI headers such as following differ, and what if this leads to runtime issues in case #3
-videodev2.h: This header is enhanced in 5.4
-fsverity.h: this header is not present in 4.19--
Regards,
Sourabh--
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
Quoted 25 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
*Kernel Recipe:* linux-yocto_5.4.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
There is no conflict in this case as libc-headers and linux-yocto
are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
*Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as
5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may lead
to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
What's the recommendation here? Should the BSP provider port
linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"
Quoted 5 lines
I suspect this as UAPI headers such as following differ, and what
if this leads to runtime issues in case #3
-videodev2.h: This header is enhanced in 5.4
-fsverity.h: this header is not present in 4.19
right. all this combos should work fine in general case, however they all are not tested with equal coverage. So best is always to try have UAPIs older or equal to the kernel you intend to use. Generally releases support supported LTS kernels and that should be good enough, however if you have very old kernel and want to use latest yocto release, be aware that we are not testing it with those old UAPIs so all testing falls on your in this regard.
Quoted 7 lines
Signature · 2 lines
Regards, Sourabh
Thank you for sharing your thoughts!
Let me focus a bit on the case where the BSP kernel is lower than libc. I.e., BSP layer comes with kernel 4.19 support for the machine in question.
From your replies I see following options:
Option #1)
Use linux-libc-headers_5.4 as it is from poky. BSP layer provides the lower version of kernel. There may be selective bugs for application (Such as the libnl case discussed on thread).
Deal with such applications case-by-case. By case-by-case, I mean, with a linux-vendor-headers. And make the application depend on linux-vendor-headers and look for the header under custom path?
Option #2)
Implement a linux-libc-headers_4.19 (or the lower version you are at). This way seemingly, the headers and kernel coming from BSP layer are reconciled.
But, doing so violates the "Poky" distro, as "Poky" originally came with linux-libc-headers_5.4 on Dunfell. And, that is bad idea, hence create your own higher distro layer that chooses linux-libc-headers_4.19.
All done, this approach still does not make things right!
As all testing/validation of 3rd-Party applications/3rd-Party layers that Yocto Project and different contributors have done using Poky and with the original linux-libc-headers_5.4, are lost.
The lower version of headers are exposing those 3rd-Party applications/3rd-Party to unknown issues.
It's sort of a dilemma!
The customer I am working with is worried about "Option #1" (linux-libc-headers_5.4 + lower BSP kernel). As discovery of 5.4 vs 4.19 bugs may not be easy to uncover. And then for for a case-by-case fix they will have to deviate away from linux-libc-headers, and make application recipes, header path specific fixes.
"Option #2" (linux-libc-headers_4.19 from a higher distro layer), seems like a way out, but at the cost of upending all the testing and validation done with linux-libc-headers_5.4 for upstream recipes/3rd-party layer.
Upgrading the machine from 4.19 to 5.4 is also off the table at this time.
Option #3)
Selective Backport.
I am trying to get a feel if this is even the right path?
Let linux-libc-headers_5.4 be as it is from poky. BSP layer provides the lower version of kernel. Identify and freeze on list of the UAPI headers that customer plans to use from linux-libc-headers_5.4.
Determine what is the gap between 4.19 and 5.4, backport those from 5.4 to 4.19. Do a similar exercise with 3rd-party OSS applications/frameworks/layers to be used on project and weed out cases like libnl by way of backporting from 5.4 to 4.19.
Does this seem to keep things aligned with linux-libc-headers from "Poky" and still addresses the holes that lower BSP kernel creates when working with linux-libc-headers_5.4?
Thanks for your time!
Signature · 2 lines
Regards, Sourabh
Quoted 46 lines · Khem Raj
Quoted 25 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different? For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
*Kernel Recipe:* linux-yocto_5.4.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
There is no conflict in this case as libc-headers and linux-yocto
are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
*Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as
5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may
lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
What's the recommendation here? Should the BSP provider port
linux-libc-header_4.19.bb in BSP layer and set PREFERRED_VERSION_linux-libc-headers = "4.19%"Quoted 5 lines
I suspect this as UAPI headers such as following differ, and what
if this leads to runtime issues in case #3
-videodev2.h: This header is enhanced in 5.4
-fsverity.h: this header is not present in 4.19right. all this combos should work fine in general case, however they all are not tested with equal coverage. So best is always to try have UAPIs older or equal to the kernel you intend to use. Generally releases support supported LTS kernels and that should be good enough, however if you have very old kernel and want to use latest yocto release, be aware that we are not testing it with those old UAPIs so all testing falls on your in this regard.
Quoted 6 lines
-- Regards,
Sourabh
Signature · 2 lines
Regards, Sourabh
Quoted 16 lines · Sourabh Banerjee
Thank you for sharing your thoughts!
Let me focus a bit on the case where the BSP kernel is lower than libc. I.e., BSP layer comes with kernel 4.19 support for the machine in question.
From your replies I see following options:
Option #1)
Use linux-libc-headers_5.4 as it is from poky. BSP layer provides the lower version of kernel.
There may be selective bugs for application (Such as the libnl case discussed on thread).
Quoted 4 lines
Deal with such applications case-by-case. By case-by-case, I mean, with a linux-vendor-headers.
And make the application depend on linux-vendor-headers and look for the header under custom path?
To be clear, you don't have to do this for everything. Everything in oe-ore (as an example) has already been tested against the libc-headers in the release, at both a lower and matching kernel version level.
It is good to be concerned, but this is not a common occurrence, and really isn't the case to optimize for (i.e. that there is an issue, most applications simply don't care, and it isn't impossible to identify ones that do).
Quoted 39 lines
Option #2)
Implement a linux-libc-headers_4.19 (or the lower version you are at). This way seemingly, the headers and kernel coming from BSP layer are reconciled.But, doing so violates the "Poky" distro, as "Poky" originally came with linux-libc-headers_5.4 on Dunfell.
And, that is bad idea, hence create your own higher distro layer that chooses linux-libc-headers_4.19.All done, this approach still does not make things right!
As all testing/validation of 3rd-Party applications/3rd-Party layers that Yocto Project
and different contributors have done using Poky and with the original linux-libc-headers_5.4, are lost.The lower version of headers are exposing those 3rd-Party applications/3rd-Party to unknown issues.
It's sort of a dilemma!
The customer I am working with is worried about "Option #1" (linux-libc-headers_5.4 + lower BSP kernel).
As discovery of 5.4 vs 4.19 bugs may not be easy to uncover. And then for for a case-by-case fix they will
have to deviate away from linux-libc-headers, and make application recipes, header path specific fixes."Option #2" (linux-libc-headers_4.19 from a higher distro layer), seems like a way out, but at the cost of upending all
the testing and validation done with linux-libc-headers_5.4 for upstream recipes/3rd-party layer.Upgrading the machine from 4.19 to 5.4 is also off the table at this time.
Quoted 8 lines
Option #3)
Selective Backport.I am trying to get a feel if this is even the right path? Let linux-libc-headers_5.4 be as it is from poky. BSP layer provides the lower version of kernel.
Identify and freeze on list of the UAPI headers that customer plans to use from linux-libc-headers_5.4.
This really isn't any different than option #1. If you can identify the UAPI headers you are interested in, and can identify the applications that use them .. then simply provide those headers to those applications as you would in #1.
I wouldn't recommend it.
If you are really concerned about the mismatch, then you can/should define your own distro and provide the matching libc-headers. That's the less error prone way to do it, if it is something you want to do.
Bruce
Quoted 78 lines
Determine what is the gap between 4.19 and 5.4, backport those from 5.4 to 4.19.
Do a similar exercise with 3rd-party OSS applications/frameworks/layers to be used on project and
weed out cases like libnl by way of backporting from 5.4 to 4.19.Does this seem to keep things aligned with linux-libc-headers from "Poky" and still addresses the holes that lower
BSP kernel creates when working with linux-libc-headers_5.4?Thanks for your time!
--
Regards,
SourabhQuoted 37 lines · Khem Raj
Quoted 30 lines · Sourabh Banerjee
Hi All,
I need your suggestion on how to reconcile if the linux-libc-headers and kernel versions are different?
For this discussion let's consider we are using the LTS branch YP-3.1 (Dunfell).
With Dunfell let's consider following 3 cases:1) Machine is supported on 5.4 kernel
*Kernel Recipe:* linux-yocto_5.4.bb
*libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
There is no conflict in this case as libc-headers and linux-yocto
are on same version.2) Machine is supported on a higher kernel (let's say 5.10)
*Kernel Recipe:* Let's assume provided by a BSP layer, linux_5.10.bb *libc-headers:* meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is higher in this case. But, should be okay as
5.10 UAPI is backward compatible and supports linux-libc-headers_5.4 completely.3) Machine is supported on a lower kernel (let's say 4.19)
Kernel Recipe: Let's assume provided by a BSP layer, linux_4.19.bb
libc-headers: meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.4.bb
The kernel version is lower in this case. I suppose, this may
lead to some runtime issues? As the kernel built from 4.19 will not be able to support 5.4 UAPI fully.
What's the recommendation here? Should the BSP provider port
linux-libc-header_4.19.bb in BSP layer and
set PREFERRED_VERSION_linux-libc-headers = "4.19%"Quoted 5 lines
I suspect this as UAPI headers such as following differ, and what
if this leads to runtime issues in case #3
-videodev2.h: This header is enhanced in 5.4
-fsverity.h: this header is not present in 4.19Quoted 14 lines
right. all this combos should work fine in general case, however they all are not tested with equal coverage. So best is always to try have UAPIs older or equal to the kernel you intend to use. Generally releases support supported LTS kernels and that should be good enough, however if you have very old kernel and want to use latest yocto release, be aware that we are not testing it with those old UAPIs so all testing falls on your in this regard.
Quoted 6 lines
-- Regards,
Sourabh--
Regards,
Sourabh
Signature · 3 lines
- Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II