Single zImage and A15/LPAE
I'm curious what the single-zImage story is for Cortex A15 CPUs with LPAE extensions; IIRC, LPAE entails a different page table format and so isn't going to co-exist in the same zImage as non-LPAE (although doesn't x86 support that now; I though separate LPAE/non-LPAE kernels went away there?)
Presumably though, the intent is to be able to disable LPAE in Kconfig, and create a single zImage with both A9 and A15 support, or to enable LPAE and only support A15 with LPAE, and which you'd choose to do simply depends on whether you care about LPAE based on how much RAM the boards you want to support have (or perhaps also what physical address your peripherals are at)?
Quoted 3 lines · Stephen Warren
I'm curious what the single-zImage story is for Cortex A15 CPUs with LPAE extensions; IIRC, LPAE entails a different page table format and so isn't going to co-exist in the same zImage as non-LPAE
LPAE vs non LPAE is an even more invasive change than ARMv6+ vs pre ARMv6 support. So no, I don't think we'll ever support LPAE and non LPAE configs in the same kernel binary.
Quoted 2 lines
(although doesn't x86 support that now; I though separate LPAE/non-LPAE kernels went away there?)
I don't think so. At least Ubuntu apparently still carries a PAE and non PAE kernel packages. Fedora doesn't, probably because they decided not to support non PAE capable machines anymore. We certainly cannot make this choice on ARM yet.
Quoted 6 lines
Presumably though, the intent is to be able to disable LPAE in
Kconfig, and create a single zImage with both A9 and A15 support, or to enable LPAE and only support A15 with LPAE, and which you'd choose to do simply depends on whether you care about LPAE based on how much RAM the boards you want to support have (or perhaps also what physical address your peripherals are at)?
Yes.
Nicolas
Quoted 10 lines · On Monday 08 October 2012 22:18:24 Nicolas Pitre wrote:
Quoted 3 lines · Stephen Warren
I'm curious what the single-zImage story is for Cortex A15 CPUs with LPAE extensions; IIRC, LPAE entails a different page table format and so isn't going to co-exist in the same zImage as non-LPAE
LPAE vs non LPAE is an even more invasive change than ARMv6+ vs pre ARMv6 support. So no, I don't think we'll ever support LPAE and non LPAE configs in the same kernel binary.
Quoted 7 lines
Quoted 2 lines
(although doesn't x86 support that now; I though separate LPAE/non-LPAE kernels went away there?)
I don't think so. At least Ubuntu apparently still carries a PAE and non PAE kernel packages. Fedora doesn't, probably because they decided not to support non PAE capable machines anymore. We certainly cannot make this choice on ARM yet.
Fedora 18 still has both PAE and non-PAE kernels. I would really hope they could give up the PAE version in favor of a 64 bit kernel in the 32 bit distro, but it seems none of the big distros trust the compat code enough yet. On x86, the number of 32 bit machines still running with more than 3GB of RAM installed should be very marginal now, most people running the PAE kernel actually have 64 bit capable CPUs and have some legacy 32 bit applications that are easier to run with a 32 bit user space.
Maybe we get to the same point on ARM in some 10 years, but for the forseeable future, Cortex-a15 machines with lots of RAM will be very real and we need to have separate kernels for those.
Arnd
Quoted 28 lines · Arnd Bergmann
Quoted 10 lines · On Monday 08 October 2012 22:18:24 Nicolas Pitre wrote:
Quoted 3 lines · Stephen Warren
I'm curious what the single-zImage story is for Cortex A15 CPUs with LPAE extensions; IIRC, LPAE entails a different page table format and so isn't going to co-exist in the same zImage as non-LPAE
LPAE vs non LPAE is an even more invasive change than ARMv6+ vs pre ARMv6 support. So no, I don't think we'll ever support LPAE and non LPAE configs in the same kernel binary.
Quoted 7 lines
Quoted 2 lines
(although doesn't x86 support that now; I though separate LPAE/non-LPAE kernels went away there?)
I don't think so. At least Ubuntu apparently still carries a PAE and non PAE kernel packages. Fedora doesn't, probably because they decided not to support non PAE capable machines anymore. We certainly cannot make this choice on ARM yet.
Fedora 18 still has both PAE and non-PAE kernels. I would really hope they could give up the PAE version in favor of a 64 bit kernel in the 32 bit distro, but it seems none of the big distros trust the compat code enough yet. On x86, the number of 32 bit machines still running with more than 3GB of RAM installed should be very marginal now, most people running the PAE kernel actually have 64 bit capable CPUs and have some legacy 32 bit applications that are easier to run with a 32 bit user space.
Quoted 3 lines
Maybe we get to the same point on ARM in some 10 years, but for the forseeable future, Cortex-a15 machines with lots of RAM will be very real and we need to have separate kernels for those.
In the medium term we could work around this with a fat kernel in principle (i.e., bundle a non-LPAE single kernel together with an LPAE one and choose the right one at boot time). This can be solved outside the kernel if necessary.
This is not great, but the general feeling seems to be that combining LPAE and non-LPAE in the same kernel is just not going to be worth the pain (either in code or runtime impacts).
For AArch64, we obviously have a different kernel binary anyway.
Cheers
---Dave
Quoted 40 lines · Dave Martin
Quoted 5 lines · Arnd Bergmann
Quoted 4 lines · On Monday 08 October 2012 22:18:24 Nicolas Pitre wrote:
Quoted 2 lines · Stephen Warren
I'm curious what the single-zImage story is for Cortex A15 CPUs with LPAE extensions; IIRC, LPAE entails a different page table format
and so
Quoted 11 lines
Quoted 11 lines
Quoted 1 line
isn't going to co-exist in the same zImage as non-LPAE
LPAE vs non LPAE is an even more invasive change than ARMv6+ vs pre ARMv6 support. So no, I don't think we'll ever support LPAE and non LPAE configs in the same kernel binary.
Quoted 2 lines
(although doesn't x86 support that now; I though separate LPAE/non-LPAE kernels went away there?)
I don't think so. At least Ubuntu apparently still carries a PAE and non PAE kernel packages. Fedora doesn't, probably because they
decided
Quoted 12 lines
Quoted 2 lines
not to support non PAE capable machines anymore. We certainly cannot make this choice on ARM yet.
Fedora 18 still has both PAE and non-PAE kernels. I would really hope they could give up the PAE version in favor of a 64 bit kernel in the 32 bit distro, but it seems none of the big distros trust the compat code enough yet. On x86, the number of 32 bit machines still running with more than 3GB of RAM installed should be very marginal now, most people running the PAE kernel actually have 64 bit capable CPUs and have some legacy 32 bit applications that are easier to run with a 32 bit user space.
Quoted 3 lines
Maybe we get to the same point on ARM in some 10 years, but for the forseeable future, Cortex-a15 machines with lots of RAM will be very real and we need to have separate kernels for those.
In the medium term we could work around this with a fat kernel in principle (i.e., bundle a non-LPAE single kernel together with an LPAE one and choose the right one at boot time). This can be solved outside the kernel if necessary.
Quoted 6 lines
This is not great, but the general feeling seems to be that combining LPAE and non-LPAE in the same kernel is just not going to be worth the pain (either in code or runtime impacts).
For AArch64, we obviously have a different kernel binary anyway.
(+ Olof, Russell)
Just note, I found this in my old mail-box...
So I think, as a result, we need to support separate kernel binary for LPAE and non-LPAE and you guys don't hold a different view about it.
Current SSDK5440 and SD5v1 boards which are reference boards for EXYNOS5440 has over 4GB memory as a default and that's why I'm writing this e-mail. It means I should keep separated kernel to support exynos5440 and other exynos SoCs.
If you have any concerns about that, please let me know.
- Kukjin
Quoted 45 lines · Kukjin Kim
Quoted 44 lines · Dave Martin
Quoted 5 lines · Arnd Bergmann
Quoted 4 lines · On Monday 08 October 2012 22:18:24 Nicolas Pitre wrote:
Quoted 2 lines · Stephen Warren
I'm curious what the single-zImage story is for Cortex A15 CPUs with LPAE extensions; IIRC, LPAE entails a different page table format
and so
Quoted 11 lines
Quoted 11 lines
Quoted 1 line
isn't going to co-exist in the same zImage as non-LPAE
LPAE vs non LPAE is an even more invasive change than ARMv6+ vs pre ARMv6 support. So no, I don't think we'll ever support LPAE and non LPAE configs in the same kernel binary.
Quoted 2 lines
(although doesn't x86 support that now; I though separate LPAE/non-LPAE kernels went away there?)
I don't think so. At least Ubuntu apparently still carries a PAE and non PAE kernel packages. Fedora doesn't, probably because they
decided
Quoted 12 lines
Quoted 2 lines
not to support non PAE capable machines anymore. We certainly cannot make this choice on ARM yet.
Fedora 18 still has both PAE and non-PAE kernels. I would really hope they could give up the PAE version in favor of a 64 bit kernel in the 32 bit distro, but it seems none of the big distros trust the compat code enough yet. On x86, the number of 32 bit machines still running with more than 3GB of RAM installed should be very marginal now, most people running the PAE kernel actually have 64 bit capable CPUs and have some legacy 32 bit applications that are easier to run with a 32 bit user space.
Quoted 3 lines
Maybe we get to the same point on ARM in some 10 years, but for the forseeable future, Cortex-a15 machines with lots of RAM will be very real and we need to have separate kernels for those.
In the medium term we could work around this with a fat kernel in principle (i.e., bundle a non-LPAE single kernel together with an LPAE one and choose the right one at boot time). This can be solved outside the kernel if necessary.
This is not great, but the general feeling seems to be that combining LPAE and non-LPAE in the same kernel is just not going to be worth the pain (either in code or runtime impacts).
Quoted 8 lines
Quoted 2 lines
For AArch64, we obviously have a different kernel binary anyway.
(+ Olof, Russell)
Just note, I found this in my old mail-box...
So I think, as a result, we need to support separate kernel binary for LPAE and non-LPAE and you guys don't hold a different view about it.
LPAE and non-LPAE configurations will most likely never be supported together in the same kernel binary. The runtime cost would simply be too high to be worth it.
Quoted 4 lines
Current SSDK5440 and SD5v1 boards which are reference boards for EXYNOS5440 has over 4GB memory as a default and that's why I'm writing this e-mail. It means I should keep separated kernel to support exynos5440 and other exynos SoCs.
They could all be supported together but with less than 4GB of memory.
Nicolas
Quoted 12 lines · Nicolas Pitre
Quoted 6 lines · Kukjin Kim
(+ Olof, Russell)
Just note, I found this in my old mail-box...
So I think, as a result, we need to support separate kernel binary for LPAE and non-LPAE and you guys don't hold a different view about it.
LPAE and non-LPAE configurations will most likely never be supported together in the same kernel binary. The runtime cost would simply be too high to be worth it.
Quoted 6 lines
Quoted 4 lines
Current SSDK5440 and SD5v1 boards which are reference boards for EXYNOS5440 has over 4GB memory as a default and that's why I'm writing this e-mail. It means I should keep separated kernel to support exynos5440 and other exynos SoCs.
They could all be supported together but with less than 4GB of memory.
Or the other way around, the other EXYNOS5 CPUs could all enable LPAE even if they have less than 4GB of memory -- but not EXYNOS4, of course.
However, instead of forking off and duplicating otherwise near-identical defconfigs, I suggest just getting your developers used to passing in CONFIG_LPAE=y on the build command line instead (or in their local configs for their products, of course). That way there's less duplication and no need to enable drivers in two places, etc.
-Olof
Quoted 24 lines · Olof Johansson
Quoted 12 lines · Nicolas Pitre
Quoted 6 lines · Kukjin Kim
(+ Olof, Russell)
Just note, I found this in my old mail-box...
So I think, as a result, we need to support separate kernel binary for LPAE and non-LPAE and you guys don't hold a different view about it.
LPAE and non-LPAE configurations will most likely never be supported together in the same kernel binary. The runtime cost would simply be too high to be worth it.
Quoted 6 lines
Quoted 4 lines
Current SSDK5440 and SD5v1 boards which are reference boards for EXYNOS5440 has over 4GB memory as a default and that's why I'm writing this e-mail. It means I should keep separated kernel to support exynos5440 and other exynos SoCs.
They could all be supported together but with less than 4GB of memory.
Or the other way around, the other EXYNOS5 CPUs could all enable LPAE even if they have less than 4GB of memory -- but not EXYNOS4, of course.
Quoted 5 lines
However, instead of forking off and duplicating otherwise near-identical defconfigs, I suggest just getting your developers used to passing in
CONFIG_LPAE=yon the build command line instead (or in their local configs for their products, of course). That way there's less duplication and no need to enable drivers in two places, etc.
Nevermind, I'm smoking crack, that doesn't seem to actually work. (Besides, it's CONFIG_ARM_LPAE).
-Olof
Quoted 18 lines · On Tuesday 28 May 2013 19:02:48 Olof Johansson wrote:
Quoted 17 lines · Nicolas Pitre
Quoted 6 lines · Kukjin Kim
(+ Olof, Russell)
Just note, I found this in my old mail-box...
So I think, as a result, we need to support separate kernel binary for LPAE and non-LPAE and you guys don't hold a different view about it.
LPAE and non-LPAE configurations will most likely never be supported together in the same kernel binary. The runtime cost would simply be too high to be worth it.
Quoted 4 lines
Current SSDK5440 and SD5v1 boards which are reference boards for EXYNOS5440 has over 4GB memory as a default and that's why I'm writing this e-mail. It means I should keep separated kernel to support exynos5440 and other exynos SoCs.
Quoted 5 lines
Quoted 1 line
They could all be supported together but with less than 4GB of memory.
Or the other way around, the other EXYNOS5 CPUs could all enable LPAE even if they have less than 4GB of memory -- but not EXYNOS4, of course.
The one bug I see is that it's currently possible to build a kernel for a Cortex-A5/A8/A9, Scorpion or PJ4 platform with LPAE enabled, which obviously cannot work.
I think to do this right, we need to treat ARMv7+LPAE as a separate architecture level, just like we treat ARMv6, ARMv7 as separate compatible architectures. BTW, is there an official name for that architecture?
If we introduce a new CONFIG_ARCH_MULTI_V7_LPAE option, that should be used instead of CONFIG_ARCH_MULTI_V7 for platforms that can be built as part of CONFIG_ARCH_MULTIPLATFORM and with LPAE support. The default multi_v7_defconfig should enable all both ARCH_MULTI_V7 and ARCH_MULTI_V7_LPAE, so we get e.g. OMAP4/OMAP5 adn Exynos4/Exynos5 in one kernel, but when you want to enable LPAE, you have to disable ARCH_MULTI_V7 and only get OMAP5 and Exynos5 out of those.
Arnd
Quoted 25 lines · Arnd Bergmann
Quoted 18 lines · On Tuesday 28 May 2013 19:02:48 Olof Johansson wrote:
Quoted 17 lines · Nicolas Pitre
Quoted 6 lines · Kukjin Kim
(+ Olof, Russell)
Just note, I found this in my old mail-box...
So I think, as a result, we need to support separate kernel binary for LPAE and non-LPAE and you guys don't hold a different view about it.
LPAE and non-LPAE configurations will most likely never be supported together in the same kernel binary. The runtime cost would simply be too high to be worth it.
Quoted 4 lines
Current SSDK5440 and SD5v1 boards which are reference boards for EXYNOS5440 has over 4GB memory as a default and that's why I'm writing this e-mail. It means I should keep separated kernel to support exynos5440 and other exynos SoCs.
Quoted 5 lines
Quoted 1 line
They could all be supported together but with less than 4GB of memory.
Or the other way around, the other EXYNOS5 CPUs could all enable LPAE even if they have less than 4GB of memory -- but not EXYNOS4, of course.
Quoted 8 lines
The one bug I see is that it's currently possible to build a kernel for a Cortex-A5/A8/A9, Scorpion or PJ4 platform with LPAE enabled, which obviously cannot work.
I think to do this right, we need to treat ARMv7+LPAE as a separate architecture level, just like we treat ARMv6, ARMv7 as separate compatible architectures. BTW, is there an official name for that architecture?
The official is Large Physical Address Extension which is an extension to ARMv7, so not a new architecture.
Signature · 1 line
Catalin
Quoted 7 lines · Catalin Marinas
Quoted 4 lines
I think to do this right, we need to treat ARMv7+LPAE as a separate architecture level, just like we treat ARMv6, ARMv7 as separate compatible architectures. BTW, is there an official name for that architecture?
The official is Large Physical Address Extension which is an extension to ARMv7, so not a new architecture.
Ah, too bad. I had hoped there were still postfixes like v6K or v4T. I guess there is no complete list of ARMv7 extensions anywhere, right?
Arnd
Quoted 11 lines · Arnd Bergmann
Quoted 7 lines · Catalin Marinas
Quoted 4 lines
I think to do this right, we need to treat ARMv7+LPAE as a separate architecture level, just like we treat ARMv6, ARMv7 as separate compatible architectures. BTW, is there an official name for that architecture?
The official is Large Physical Address Extension which is an extension to ARMv7, so not a new architecture.
Ah, too bad. I had hoped there were still postfixes like v6K or v4T. I guess there is no complete list of ARMv7 extensions anywhere, right?
It's the ARM ARM, though not a list. With ARMv7 we moved away from adding suffixes to the architecture number and introduced the CPUID scheme to check for what's available on a CPU.
Signature · 1 line
Catalin
Quoted 7 lines · Arnd Bergmann
Quoted 5 lines · Catalin Marinas
Quoted 4 lines
I think to do this right, we need to treat ARMv7+LPAE as a separate architecture level, just like we treat ARMv6, ARMv7 as separate compatible architectures. BTW, is there an official name for that architecture?
Quoted 6 lines
Quoted 2 lines
The official is Large Physical Address Extension which is an extension to ARMv7, so not a new architecture.
Ah, too bad. I had hoped there were still postfixes like v6K or v4T. I guess there is no complete list of ARMv7 extensions anywhere, right?
Just note, EXYNOS4 SoCs and EXYNOS5250 cannot boot when LPAE is enabled so I think, single zImage should be handled separately for non-LPAE and LPAE. I need to sort out EXYNOS SoCs for it and prepare registering exynos5440_defconfig. Of course, EXYNOS5440 can be supported without LPAE enabling but current bootargs in v3.11-rc1 break the kernel booting.
Thanks,
- Kukjin