Skip to contentSkip to conversation

arm64: Document requirements for fine grained traps at boot

PATCH Superseded Mark Brown 3 participants 6 messages Mar 12, 2021 – Mar 29, 2021 Documentation

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Mark Brown

author
#0 lore raw

PATCH 1/1 arm64: Document requirements for fine grained traps at boot

The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses to small sets of registers and instructions from EL1 and EL0. Currently Linux makes no use of this feature, explicitly document that it should be disabled when entering the kernel at EL2 (as is the architectural default) to help avoid surprises.

---
 Documentation/arm64/booting.rst | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.rst +7 -0
index 7552dbc1cc54..1efc2d3023bb 100644
@@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
       having 0b1 set for the corresponding bit for each of the auxiliary
       counters present.
 
+  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
+
+  - If the kernel is entered at EL2:
+
+    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
+      HFGITR_EL2 must be initialised to 0.
+
 The requirements described above for CPU mode, caches, MMUs, architected
 timers, coherency and system registers apply to all CPUs.  All CPUs must
 enter the kernel in the same exception level.
Signature · 7 lines
2.20.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#1 lore raw
Quoted 16 lines · Mark Brown

The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses to small sets of registers and instructions from EL1 and EL0. Currently Linux makes no use of this feature, explicitly document that it should be disabled when entering the kernel at EL2 (as is the architectural default) to help avoid surprises.

---
 Documentation/arm64/booting.rst | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.rst +0 -0
index 7552dbc1cc54..1efc2d3023bb 100644

Documentation/arm64/booting.rst Before jumping into the kernel, the following conditions must be met: · line 278

Quoted 9 lines
Documentation/arm64/booting.rst +6 -0
@@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
       having 0b1 set for the corresponding bit for each of the auxiliary
       counters present.
 
+  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
+
+  - If the kernel is entered at EL2:
+
+    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
+      HFGITR_EL2 must be initialised to 0.

While this requirement is correct, documenting such individual registers doesn't scales well. You may run a 5 year old kernel on a newer CPU and we can't predict which control registers have been added and what side-effect they have. The architecture, at least for the above registers, states that if warm reset to EL2, their value is 0. I think the EL3 firmware (which is normally up to date with the CPU it is running on) should follow the ARM ARM reset values. There are probably EL1 registers with similar requirements (I haven't checked).

Can we instead have a broad statement regarding any EL1/EL2 registers that they should be either rest to 0 or to the architectural (warm) reset value as per the ARM ARM? Or something like any feature must be disabled by default at the EL1/EL2 control registers level and this would imply the fine-grained traps.

We currently have this statement:

All writable architected system registers at the exception level where the kernel image will be entered must be initialised by software at a higher exception level to prevent execution in an UNKNOWN state.

The "prevent execution in an UNKNOWN state" needs to be clearer. The above should also include exception levels _below_ the one where the kernel is entered. It doesn't help if KVM is old and has no clue of new EL1-specific registers.

Adding Mark R, I think he looked at some of these in the past.

Signature · 6 lines
Catalin

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#2 lore raw
Quoted 20 lines · Catalin Marinas
Quoted 19 lines · Mark Brown

The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses to small sets of registers and instructions from EL1 and EL0. Currently Linux makes no use of this feature, explicitly document that it should be disabled when entering the kernel at EL2 (as is the architectural default) to help avoid surprises.

---
 Documentation/arm64/booting.rst | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.rst +0 -0
index 7552dbc1cc54..1efc2d3023bb 100644
@@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
       having 0b1 set for the corresponding bit for each of the auxiliary
       counters present.
 
Quoted 15 lines

Documentation/arm64/booting.rst Before jumping into the kernel, the following conditions must be met: · line 278

Quoted 6 lines
Documentation/arm64/booting.rst +6 -0
+  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
+
+  - If the kernel is entered at EL2:
+
+    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
+      HFGITR_EL2 must be initialised to 0.

While this requirement is correct, documenting such individual registers doesn't scales well. You may run a 5 year old kernel on a newer CPU and we can't predict which control registers have been added and what side-effect they have. The architecture, at least for the above registers, states that if warm reset to EL2, their value is 0. I think the EL3 firmware (which is normally up to date with the CPU it is running on) should follow the ARM ARM reset values. There are probably EL1 registers with similar requirements (I haven't checked).

One thing I don't understand about the registers listed here is that we're requiring firmware to initialise them when the kernel is entered at EL2. But they're *_EL2 registers, so why can't the kernel initialise them itself? The fewer dependencies on firmware, the better.

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#3 lore raw
Quoted 37 lines · Will Deacon
Quoted 20 lines · Catalin Marinas
Quoted 19 lines · Mark Brown

The arm64 FEAT_FGT extension introduces a set of traps to EL2 for accesses to small sets of registers and instructions from EL1 and EL0. Currently Linux makes no use of this feature, explicitly document that it should be disabled when entering the kernel at EL2 (as is the architectural default) to help avoid surprises.

---
 Documentation/arm64/booting.rst | 7 +++++++
 1 file changed, 7 insertions(+)
Documentation/arm64/booting.rst +0 -0
index 7552dbc1cc54..1efc2d3023bb 100644
@@ -270,6 +270,13 @@ Before jumping into the kernel, the following conditions must be met:
       having 0b1 set for the corresponding bit for each of the auxiliary
       counters present.
 
Quoted 15 lines

Documentation/arm64/booting.rst Before jumping into the kernel, the following conditions must be met: · line 278

Quoted 6 lines
Documentation/arm64/booting.rst +6 -0
+  For CPUs with Fine Grained Traps (FEAT_FGT) extension present:
+
+  - If the kernel is entered at EL2:
+
+    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
+      HFGITR_EL2 must be initialised to 0.

While this requirement is correct, documenting such individual registers doesn't scales well. You may run a 5 year old kernel on a newer CPU and we can't predict which control registers have been added and what side-effect they have. The architecture, at least for the above registers, states that if warm reset to EL2, their value is 0. I think the EL3 firmware (which is normally up to date with the CPU it is running on) should follow the ARM ARM reset values. There are probably EL1 registers with similar requirements (I haven't checked).

Quoted 4 lines

One thing I don't understand about the registers listed here is that we're requiring firmware to initialise them when the kernel is entered at EL2. But they're *_EL2 registers, so why can't the kernel initialise them itself? The fewer dependencies on firmware, the better.

W.r.t. these specific registers, you are right, we'd better initialise them in the kernel rather than documenting. But in the general case, there may be new registers an old kernel is not aware of. Here we expect at least the EL3 firmware to be up to date with the CPU it is running on, hence a broad statement that such registers need to be initialised (unless we have one already).

Signature · 6 lines
Catalin

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
author
#4 lore raw
In reply to Will Deacon
Quoted 4 lines · Will Deacon

One thing I don't understand about the registers listed here is that we're requiring firmware to initialise them when the kernel is entered at EL2. But they're *_EL2 registers, so why can't the kernel initialise them itself? The fewer dependencies on firmware, the better.

Well, there's nothing stopping us doing both - that's the most robust thing. Relying on the kernel alone to do this has the old kernels on new systems issue, and there's always going to be some section of code running before we manage to go through and enumerate features then reset the system registers even if we run that code very early. With something like traps it seems especially relevant, it's more of a "could affect anything" kind of thing than something where we can initialize the feature before we start using it.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author
#5 lore raw
In reply to Catalin Marinas
Quoted 9 lines · Catalin Marinas

Documentation/arm64/booting.rst Before jumping into the kernel, the following conditions must be met: · line 278

Quoted 2 lines · Mark Brown
Documentation/arm64/booting.rst +2 -0
+    - HAFGRTR_EL2, HDFGWTR_EL2, HDFGRTR_EL2, HFGWTR_EL2, HFGRTR_EL2 and
+      HFGITR_EL2 must be initialised to 0.

While this requirement is correct, documenting such individual registers doesn't scales well. You may run a 5 year old kernel on a newer CPU and we can't predict which control registers have been added and what side-effect they have. The architecture, at least for the above

I'm not a huge fan of writing things this way either, I'd inferred that this sort of minimal and specific thing was the idiom desired so was trying to follow.

Quoted 8 lines

Can we instead have a broad statement regarding any EL1/EL2 registers that they should be either rest to 0 or to the architectural (warm) reset value as per the ARM ARM? Or something like any feature must be disabled by default at the EL1/EL2 control registers level and this would imply the fine-grained traps.

We currently have this statement:

Quoted 8 lines

All writable architected system registers at the exception level where the kernel image will be entered must be initialised by software at a higher exception level to prevent execution in an UNKNOWN state.

The "prevent execution in an UNKNOWN state" needs to be clearer. The above should also include exception levels _below_ the one where the kernel is entered. It doesn't help if KVM is old and has no clue of new EL1-specific registers.

Right, I think both those things are needed to cover the general case. The exception levels bit is hopefully straightforward but wording for the clarification bit is more tricky - I'll try to come up with something.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel