arm64 head.S fixes
cover arm64 head.S fixes
From: Ard Biesheuvel <[email protected]>
Ensure the MMU is off before changing HCR_EL2.E2H when entering at EL2. Drop MMU disable workarounds where they are not needed.
Ard Biesheuvel (2):
arm64/head: Drop unnecessary pre-disable-MMU workaround arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H
arch/arm64/kernel/head.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
Signature · 7 lines
2.44.0.683.g7961c838ac-goog _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 1/2 arm64/head: Drop unnecessary pre-disable-MMU workaround
From: Ard Biesheuvel <[email protected]>
The Falkor erratum that results in the need for an ISB before clearing the M bit in SCTLR_ELx only applies to execution at exception level x, and so the workaround is not needed when disabling the EL1 MMU while running at EL2.
--- arch/arm64/kernel/head.S | 2 -- 1 file changed, 2 deletions(-)
arch/arm64/kernel/head.S +0 -2
@@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) cbz x0, 2f /* Set a sane SCTLR_EL1, the VHE way */ - pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x1 mov x2, #BOOT_CPU_FLAG_E2H b 3f 2: - pre_disable_mmu_workaround msr sctlr_el1, x1 mov x2, xzr 3:
Signature · 7 lines
2.44.0.683.g7961c838ac-goog _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround
Quoted 21 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
The Falkor erratum that results in the need for an ISB before clearing the M bit in
SCTLR_ELxonly applies to execution at exception level x, and so the workaround is not needed when disabling the EL1 MMU while running at EL2.Signed-off-by Ard Biesheuvel <[email protected]>--- arch/arm64/kernel/head.S | 2 -- 1 file changed, 2 deletions(-)arch/arm64/kernel/head.S +0
-0@@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) cbz x0, 2f /* Set a sane SCTLR_EL1, the VHE way */
arch/arm64/kernel/head.S SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) · line 333
Quoted 10 lines
arch/arm64/kernel/head.S +0
-2- pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x1 mov x2, #BOOT_CPU_FLAG_E2H b 3f 2: - pre_disable_mmu_workaround msr sctlr_el1, x1 mov x2, xzr 3:
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround
In reply to Ard BiesheuvelQuoted 8 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
The Falkor erratum that results in the need for an ISB before clearing the M bit in
SCTLR_ELxonly applies to execution at exception level x, and so the workaround is not needed when disabling the EL1 MMU while running at EL2.Signed-off-by Ard Biesheuvel <[email protected]>
Mark.
Quoted 25 lines
--- arch/arm64/kernel/head.S | 2 -- 1 file changed, 2 deletions(-)arch/arm64/kernel/head.S +0
-2@@ -323,13 +323,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) cbz x0, 2f /* Set a sane SCTLR_EL1, the VHE way */ - pre_disable_mmu_workaround msr_s SYS_SCTLR_EL12, x1 mov x2, #BOOT_CPU_FLAG_E2H b 3f 2: - pre_disable_mmu_workaround msr sctlr_el1, x1 mov x2, xzr 3:Signature · 1 line
2.44.0.683.g7961c838ac-goog
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 2/2 arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Even though the boot protocol stipulates otherwise, an exception has been made for the EFI stub, and entering the core kernel with the MMU enabled is permitted. This allows a substantial amount of cache maintenance to be elided, wich is significant when fast boot times are critical (e.g., for booting micro-VMs)
Once the initial ID map has been populated, the MMU is disabled as part of the logic sequence that puts all system registers into a known state. Any code that needs to execute within the window where the MMU is off is cleaned to the PoC explicitly, which includes all of HYP text when entering at EL2.
However, the current sequence of initializing the EL2 system registers is not safe: HCR_EL2 is set to its nVHE initial state before SCTLR_EL2 is reprogrammed, and this means that a VHE-to-nVHE switch may occur while the MMU is enabled. This switch causes some system registers as well as page table descriptors to be interpreted in a different way, potentially resulting in spurious exceptions relating to MMU translation.
So disable the MMU explicitly first when entering in EL2 with the MMU and caches enabled.
--- arch/arm64/kernel/head.S | 5 +++++ 1 file changed, 5 insertions(+)
arch/arm64/kernel/head.S +5 -0
@@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) adr_l x1, __hyp_text_end adr_l x2, dcache_clean_poc blr x2 + + mov_q x0, INIT_SCTLR_EL2_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb 0: mov_q x0, HCR_HOST_NVHE_FLAGS
Signature · 7 lines
2.44.0.683.g7961c838ac-goog _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H
Quoted 37 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
Even though the boot protocol stipulates otherwise, an exception has been made for the EFI stub, and entering the core kernel with the MMU enabled is permitted. This allows a substantial amount of cache maintenance to be elided, wich is significant when fast boot times are critical (e.g., for booting micro-VMs)
Once the initial ID map has been populated, the MMU is disabled as part of the logic sequence that puts all system registers into a known state. Any code that needs to execute within the window where the MMU is off is cleaned to the PoC explicitly, which includes all of HYP text when entering at EL2.
However, the current sequence of initializing the EL2 system registers is not safe:
HCR_EL2is set to its nVHE initial state beforeSCTLR_EL2is reprogrammed, and this means that a VHE-to-nVHE switch may occur while the MMU is enabled. This switch causes some system registers as well as page table descriptors to be interpreted in a different way, potentially resulting in spurious exceptions relating to MMU translation.So disable the MMU explicitly first when entering in EL2 with the MMU and caches enabled.
Signed-off-by Ard Biesheuvel <[email protected]>--- arch/arm64/kernel/head.S | 5 +++++ 1 file changed, 5 insertions(+)arch/arm64/kernel/head.S +0
-0@@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) adr_l x1, __hyp_text_end
arch/arm64/kernel/head.S SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) · line 299
Quoted 10 lines
arch/arm64/kernel/head.S +5
-0adr_l x2, dcache_clean_poc blr x2 + + mov_q x0, INIT_SCTLR_EL2_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb 0: mov_q x0, HCR_HOST_NVHE_FLAGS
M.
Signature · 6 lines
Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Re: [PATCH 2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H
In reply to Ard BiesheuvelQuoted 22 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
Even though the boot protocol stipulates otherwise, an exception has been made for the EFI stub, and entering the core kernel with the MMU enabled is permitted. This allows a substantial amount of cache maintenance to be elided, wich is significant when fast boot times are critical (e.g., for booting micro-VMs)
Once the initial ID map has been populated, the MMU is disabled as part of the logic sequence that puts all system registers into a known state. Any code that needs to execute within the window where the MMU is off is cleaned to the PoC explicitly, which includes all of HYP text when entering at EL2.
However, the current sequence of initializing the EL2 system registers is not safe:
HCR_EL2is set to its nVHE initial state beforeSCTLR_EL2is reprogrammed, and this means that a VHE-to-nVHE switch may occur while the MMU is enabled. This switch causes some system registers as well as page table descriptors to be interpreted in a different way, potentially resulting in spurious exceptions relating to MMU translation.
Quoted 4 lines
So disable the MMU explicitly first when entering in EL2 with the MMU and caches enabled.
Signed-off-by Ard Biesheuvel <[email protected]>
Mark.
Quoted 23 lines
--- arch/arm64/kernel/head.S | 5 +++++ 1 file changed, 5 insertions(+)arch/arm64/kernel/head.S +5
-0@@ -289,6 +289,11 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL) adr_l x1, __hyp_text_end adr_l x2, dcache_clean_poc blr x2 + + mov_q x0, INIT_SCTLR_EL2_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb 0: mov_q x0, HCR_HOST_NVHE_FLAGSSignature · 1 line
2.44.0.683.g7961c838ac-goog
_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 9 lines · Ard Biesheuvel
Ensure the MMU is off before changing HCR_EL2.E2H when entering at EL2. Drop MMU disable workarounds where they are not needed.
Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Marc Zyngier <[email protected]>Cc Mark Rutland <[email protected]>[...]
Applied to arm64 (for-next/fixes), thanks!
[1/2] arm64/head: Drop unnecessary pre-disable-MMU workaround
https://git.kernel.org/arm64/c/2b504e162037 [2/2] arm64/head: Disable MMU at EL2 before clearing HCR_EL2.E2H https://git.kernel.org/arm64/c/34e526cb7d46
I added a Fixes tag and cc stable on the second patch, it goes all the way to 6.3 (this should actually be 6.6 LTS).
Signature · 7 lines
Catalin _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel