arm64: booting.rst: Clarify EL2 and cover v8-R64
cover arm64: booting.rst: Clarify EL2 and cover v8-R64
Hi,
clarify the wording of the ARM64 boot protocol requirements, to both cater for the introduction of secure EL2 in Armv8.4, but also for the addition of the Armv8-R64 architecture branch.
There is an ARM ARM supplement for v8-R64[1], and also a free model[2].
On top of those clarifications, there are at least two other issues to consider for v8-R64 on Linux, I wonder if those should be documented somewhere, although I doubt that booting.rst is the right place:
- Linux clears the NS/NSTable bit in the PTEs. In non-secure world those bits are ignored, but when in secure world this means "secure PA". That luckily matches the design here (secure-only), but we should avoid re-purposing those bits in the future (which would be technically possible when running only non-secure).
- The GIC needs to be implemented using a "single Security state" for the Linux GIC driver to work. The model mentioned above defaults to a security aware GIC, so needs to be told off using the command line. I wonder if this is should be explicitly mentioned somewhere, to avoid disappointment by implementors later.
I'd be grateful for any hint whether to state those things and where to put them.
Cheers,
Andre
[1] https://developer.arm.com/documentation/ddi0600/latest/
[2] https://developer.arm.com/tools-and-software/simulation-models/fixed-virtual-platforms/arm-ecosystem-models
Andre Przywara (2):
arm64: booting.rst: Clarify on requiring non-secure EL2
arm64: booting.rst: Cover Armv8-R64
Documentation/arm64/booting.rst | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/2 arm64: booting.rst: Clarify on requiring non-secure EL2
The ARMv8.4 architecture revision introduced the EL2 exception level to the secure world. Clarify the existing wording to make sure that Linux relies on being executed in the non-secure state.
--- Documentation/arm64/booting.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Documentation/arm64/booting.rst +5 -5
@@ -10,9 +10,9 @@ This document is based on the ARM booting document by Russell King and is relevant to all public releases of the AArch64 Linux kernel. The AArch64 exception model is made up of a number of exception levels -(EL0 - EL3), with EL0 and EL1 having a secure and a non-secure -counterpart. EL2 is the hypervisor level and exists only in non-secure -mode. EL3 is the highest priority level and exists only in secure mode. +(EL0 - EL3), with EL0, EL1 and EL2 having a secure and a non-secure +counterpart. EL2 is the hypervisor level, EL3 is the highest priority +level and exists only in secure mode. Both are architecturally optional. For the purposes of this document, we will use the term `boot loader` simply to define all software that executes on the CPU(s) before control
@@ -167,8 +167,8 @@ Before jumping into the kernel, the following conditions must be met: All forms of interrupts must be masked in PSTATE.DAIF (Debug, SError, IRQ and FIQ). - The CPU must be in either EL2 (RECOMMENDED in order to have access to - the virtualisation extensions) or non-secure EL1. + The CPU must be in non-secure state, either in EL2 (RECOMMENDED in order + to have access to the virtualisation extensions), or in EL1. - Caches, MMUs
Signature · 1 line
2.25.1
Re: [PATCH 1/2] arm64: booting.rst: Clarify on requiring non-secure EL2
Quoted 27 lines · Andre Przywara
The ARMv8.4 architecture revision introduced the EL2 exception level to the secure world. Clarify the existing wording to make sure that Linux relies on being executed in the non-secure state.
Signed-off-by Andre Przywara <[email protected]>--- Documentation/arm64/booting.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)Documentation/arm64/booting.rst +3
-3@@ -10,9 +10,9 @@ This document is based on the ARM booting document by Russell King and is relevant to all public releases of the AArch64 Linux kernel. The AArch64 exception model is made up of a number of exception levels -(EL0 - EL3), with EL0 and EL1 having a secure and a non-secure -counterpart. EL2 is the hypervisor level and exists only in non-secure -mode. EL3 is the highest priority level and exists only in secure mode. +(EL0 - EL3), with EL0, EL1 and EL2 having a secure and a non-secure +counterpart. EL2 is the hypervisor level, EL3 is the highest priority +level and exists only in secure mode. Both are architecturally optional. For the purposes of this document, we will use the term `boot loader` simply to define all software that executes on the CPU(s) before control
Documentation/arm64/booting.rst Before jumping into the kernel, the following conditions must be met: · line 171
Quoted 7 lines
Documentation/arm64/booting.rst +2
-2@@ -167,8 +167,8 @@ Before jumping into the kernel, the following conditions must be met: All forms of interrupts must be masked in PSTATE.DAIF (Debug, SError, IRQ and FIQ). - The CPU must be in either EL2 (RECOMMENDED in order to have access to - the virtualisation extensions) or non-secure EL1. + The CPU must be in non-secure state, either in EL2 (RECOMMENDED in order + to have access to the virtualisation extensions), or in EL1.
^^
Nit: double space
It might be clearer to explicitly say "non-secure EL2" and "non-secure EL1" here, but either way this looks good to me, so with the whitespace fixed:
Thanks,
Mark.
PATCH 2/2 arm64: booting.rst: Cover Armv8-R64
In reply to Andre PrzywaraThere is a new revision of the ARMv8-R architecture [1], which optionally introduces kernel compatibility - by introducing an MMU into EL1 and EL0.
Linux can run on such an implementation, if it is entered in EL1 and VMSA is both implemented and enabled for that exception level.
Clarify our kernel boot protocol to make this an officially supported mode of operation, but also limit the expectations about running in secure state (which is the only security state in v8-R).
Also we heavily rely on the Virtual Memory System Architecture (VMSA), make this explicit in the text, as this allows to cover v8-R64 as well.
[1] https://developer.arm.com/documentation/ddi0600/latest/
--- Documentation/arm64/booting.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
Documentation/arm64/booting.rst +7 -2
@@ -167,8 +167,13 @@ Before jumping into the kernel, the following conditions must be met: All forms of interrupts must be masked in PSTATE.DAIF (Debug, SError, IRQ and FIQ). - The CPU must be in non-secure state, either in EL2 (RECOMMENDED in order - to have access to the virtualisation extensions), or in EL1. + If the CPU supports two security states, Linux must be entered in + non-secure state, either in EL2 (RECOMMENDED in order to have access + to the virtualisation extensions) or in EL1. + If the CPU only supports a single security state, Linux can be run even + when this single state is "secure". + The exception level the kernel is entered in must support the VMSA + memory model. - Caches, MMUs
Signature · 1 line
2.25.1
Re: [PATCH 2/2] arm64: booting.rst: Cover Armv8-R64
Quoted 25 lines · Andre Przywara
There is a new revision of the ARMv8-R architecture [1], which optionally introduces kernel compatibility - by introducing an MMU into EL1 and EL0.
Linux can run on such an implementation, if it is entered in EL1 and VMSA is both implemented and enabled for that exception level.Clarify our kernel boot protocol to make this an officially supported mode of operation, but also limit the expectations about running in secure state (which is the only security state in v8-R).
Also we heavily rely on the Virtual Memory System Architecture (VMSA), make this explicit in the text, as this allows to cover v8-R64 as well.
[1] https://developer.arm.com/documentation/ddi0600/latest/
Signed-off-by Andre Przywara <[email protected]>--- Documentation/arm64/booting.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)Documentation/arm64/booting.rst +0
-0
Documentation/arm64/booting.rst Before jumping into the kernel, the following conditions must be met: · line 174
Quoted 10 lines
Documentation/arm64/booting.rst +5
-2@@ -167,8 +167,13 @@ Before jumping into the kernel, the following conditions must be met: All forms of interrupts must be masked in PSTATE.DAIF (Debug, SError, IRQ and FIQ). - The CPU must be in non-secure state, either in EL2 (RECOMMENDED in order - to have access to the virtualisation extensions), or in EL1. + If the CPU supports two security states, Linux must be entered in + non-secure state, either in EL2 (RECOMMENDED in order to have access + to the virtualisation extensions) or in EL1. + If the CPU only supports a single security state, Linux can be run even + when this single state is "secure".
Hmm... we've never supported running on the secure side so far, so are we certain that everything actually works in such configs?
I know that some control fields (e.g. for filtering debug/tracing and so on) differ across S/NS, and IIRC there's a bunch of GIC configuration that could differ (but I could be mistaken).
Is there anything we need to have initialized differently by firmware?
Thanks,
Mark.
Quoted 8 lines
Documentation/arm64/booting.rst +2
-0+ The exception level the kernel is entered in must support the VMSA + memory model. - Caches, MMUsSignature · 1 line
2.25.1
Quoted 19 lines · Andre Przywara
clarify the wording of the ARM64 boot protocol requirements, to both cater for the introduction of secure EL2 in Armv8.4, but also for the addition of the Armv8-R64 architecture branch.
There is an ARM ARM supplement for v8-R64[1], and also a free model[2].On top of those clarifications, there are at least two other issues to consider for v8-R64 on Linux, I wonder if those should be documented somewhere, although I doubt that
booting.rstis the right place:
- Linux clears the NS/NSTable bit in the PTEs. In non-secure world those bits are ignored, but when in secure world this means "secure PA". That luckily matches the design here (secure-only), but we should avoid re-purposing those bits in the future (which would be technically possible when running only non-secure).
- The GIC needs to be implemented using a "single Security state" for the Linux GIC driver to work. The model mentioned above defaults to a security aware GIC, so needs to be told off using the command line. I wonder if this is should be explicitly mentioned somewhere, to avoid disappointment by implementors later.
Quoted 1 line
[...]
Applied first patch to arm64 (for-next/docs), thanks!
[1/2] arm64: booting.rst: Clarify on requiring non-secure EL2
https://git.kernel.org/arm64/c/b8ac4ee08d48
Cheers,
Signature · 5 lines
Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev