arm64: kernel: fix tcr_el1.t0sz restore on systems with extended idmap
PATCH 1/1 arm64: kernel: fix tcr_el1.t0sz restore on systems with extended idmap
Commit dd006da21646 ("arm64: mm: increase VA range of identity map") introduced a mechanism to extend the virtual memory map range to support arm64 systems with system RAM located at very high offset, where the identity mapping used to enable/disable the MMU requires additional translation levels to map the physical memory at an equal virtual offset.
The kernel detects at boot time the tcr_el1.t0sz value required by the identity mapping and sets-up the tcr_el1.t0sz register field accordingly, any time the identity map is required in the kernel (ie when enabling the MMU).
After enabling the MMU, in the cold boot path the kernel resets the tcr_el1.t0sz to its default value (ie the actual configuration value for the system virtual address space) so that after enabling the MMU the memory space translated by ttbr0_el1 is restored as expected.
Commit dd006da21646 ("arm64: mm: increase VA range of identity map") also added code to set-up the tcr_el1.t0sz value when the kernel resumes from low-power states with the MMU off through cpu_resume() in order to effectively use the identity mapping to enable the MMU but failed to add the code required to restore the tcr_el1.t0sz to its default value, when the core returns to the kernel with the MMU enabled, so that the kernel might end up running with tcr_el1.t0sz value set-up for the identity mapping which can be lower than the value required by the actual virtual address space, resulting in an erroneous set-up.
This patchs adds code in the resume path that restores the tcr_el1.t0sz default value upon core resume, mirroring this way the cold boot path behaviour therefore fixing the issue.
--- arch/arm64/kernel/suspend.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
arch/arm64/kernel/suspend.c +13 -9
@@ -80,17 +80,21 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))if (ret ==0 ) {/* * We are resuming from reset with TTBR0_EL1 set to the -* idmap to enable the MMU; restore the active_mm mappings in -* TTBR0_EL1 unless the active_mm == &init_mm, in which case -* the thread entered cpu_suspend with TTBR0_EL1 set to -* reserved TTBR0 page tables and should be restored as such. +* idmap to enable the MMU; set the TTBR0 to the reserved +* page tables to prevent speculative TLB allocations, flush +* the local tlb and set the default tcr_el1.t0sz so that +* the TTBR0 address space set-up is properly restored. +* If the current active_mm != &init_mm we entered cpu_suspend +* with mappings in TTBR0 that must be restored, so we switch +* them back to complete the address space configuration +* restoration before returning. */ -if (mm == &init_mm) -cpu_set_reserved_ttbr0 (); -else -cpu_switch_mm (mm->pgd, mm); - +cpu_set_reserved_ttbr0 ();flush_tlb_all (); +cpu_set_default_tcr_t0sz (); + +if (mm != &init_mm) +cpu_switch_mm (mm->pgd, mm);/* * Restore per-cpu offset before any kernel
Signature · 1 line
2.5.1
PATCH 1/1 arm64: kernel: fix tcr_el1.t0sz restore on systems with extended idmap
Hi Lorenzo,
Thanks for the fix. I'm a bit embarrassed since this is quite obviously broken, and I (nor anyone else, for that matter) spotted it through testing suspend/resume on Seattle (frankly, I don't even know if suspend/resume works currently on Seattle or if it did at the time)
Quoted 12 lines · Lorenzo Pieralisi
Commit
dd006da21646("arm64: mm: increase VA range of identity map") introduced a mechanism to extend the virtual memory map range to support arm64 systems with system RAM located at very high offset, where the identity mapping used to enable/disable the MMU requires additional translation levels to map the physical memory at an equal virtual offset.The kernel detects at boot time the tcr_el1.t0sz value required by the identity mapping and sets-up the tcr_el1.t0sz register field accordingly, any time the identity map is required in the kernel (ie when enabling the MMU).
Quoted 5 lines
After enabling the MMU, in the cold boot path the kernel resets the tcr_el1.t0sz to its default value (ie the actual configuration value for the system virtual address space) so that after enabling the MMU the memory space translated by
ttbr0_el1is restored as expected.
'After enabling the MMU' is confusing, especially since you use it twice to describe two different points in time.
Quoted 10 lines
Commit
dd006da21646("arm64: mm: increase VA range of identity map") also added code to set-up the tcr_el1.t0sz value when the kernel resumes from low-power states with the MMU off throughcpu_resume()in order to effectively use the identity mapping to enable the MMU but failed to add the code required to restore the tcr_el1.t0sz to its default value, when the core returns to the kernel with the MMU enabled, so that the kernel might end up running with tcr_el1.t0sz value set-up for the identity mapping which can be lower than the value required by the actual virtual address space, resulting in an erroneous set-up.
Quoted 4 lines
This patchs adds code in the resume path that restores the tcr_el1.t0sz default value upon core resume, mirroring this way the cold boot path behaviour therefore fixing the issue.
Of course, I agree with the patch, but could you condense the commit log, please? I don't think we need five paragraphs to point out that the sequence to set T0SZ to its idmap value, enable the MMU, branch into the kernel virtual mapping and set T0SZ to its default value is missing the final step on the resume path, and it actually took me a while to figure out what the problem was from reading the text.
Quoted 1 line
Fixes 95322526ef62 ("arm64: kernel: cpu_{suspend/resume} implementation")
Wrong commit id
Quoted 4 lines
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Will Deacon <[email protected]>Cc Catalin Marinas <[email protected]>
Quoted 44 lines
--- arch/arm64/kernel/suspend.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)arch/arm64/kernel/suspend.c +13
-9@@ -80,17 +80,21 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long))if (ret ==0 ) {/* * We are resuming from reset with TTBR0_EL1 set to the -* idmap to enable the MMU; restore the active_mm mappings in -* TTBR0_EL1 unless the active_mm == &init_mm, in which case -* the thread entered cpu_suspend with TTBR0_EL1 set to -* reserved TTBR0 page tables and should be restored as such. +* idmap to enable the MMU; set the TTBR0 to the reserved +* page tables to prevent speculative TLB allocations, flush +* the local tlb and set the default tcr_el1.t0sz so that +* the TTBR0 address space set-up is properly restored. +* If the current active_mm != &init_mm we entered cpu_suspend +* with mappings in TTBR0 that must be restored, so we switch +* them back to complete the address space configuration +* restoration before returning. */ -if (mm == &init_mm) -cpu_set_reserved_ttbr0 (); -else -cpu_switch_mm (mm->pgd, mm); - +cpu_set_reserved_ttbr0 ();flush_tlb_all (); +cpu_set_default_tcr_t0sz (); + +if (mm != &init_mm) +cpu_switch_mm (mm->pgd, mm);/* * Restore per-cpu offset before any kernel
--
2.5.1
PATCH 1/1 arm64: kernel: fix tcr_el1.t0sz restore on systems with extended idmap
Hi Ard,
Quoted 6 lines · Ard Biesheuvel
Hi Lorenzo,
Thanks for the fix. I'm a bit embarrassed since this is quite obviously broken, and I (nor anyone else, for that matter) spotted it through testing suspend/resume on Seattle (frankly, I don't even know if suspend/resume works currently on Seattle or if it did at the time)
You could not spot it by testing on Seattle, it does not implement PM yet, I spotted it by code inspection while reviewing James' suspend-to-disk patches, there is nothing to be embarassed about, you paved the way to fix it.
Quoted 13 lines · Lorenzo Pieralisi
Quoted 12 lines
Commit
dd006da21646("arm64: mm: increase VA range of identity map") introduced a mechanism to extend the virtual memory map range to support arm64 systems with system RAM located at very high offset, where the identity mapping used to enable/disable the MMU requires additional translation levels to map the physical memory at an equal virtual offset.The kernel detects at boot time the tcr_el1.t0sz value required by the identity mapping and sets-up the tcr_el1.t0sz register field accordingly, any time the identity map is required in the kernel (ie when enabling the MMU).
Quoted 8 lines
Quoted 5 lines
After enabling the MMU, in the cold boot path the kernel resets the tcr_el1.t0sz to its default value (ie the actual configuration value for the system virtual address space) so that after enabling the MMU the memory space translated by
ttbr0_el1is restored as expected.'After enabling the MMU' is confusing, especially since you use it twice to describe two different points in time.
I will sum up this commit log, it is ways too long and not clear at times.
Quoted 15 lines
Quoted 10 lines
Commit
dd006da21646("arm64: mm: increase VA range of identity map") also added code to set-up the tcr_el1.t0sz value when the kernel resumes from low-power states with the MMU off throughcpu_resume()in order to effectively use the identity mapping to enable the MMU but failed to add the code required to restore the tcr_el1.t0sz to its default value, when the core returns to the kernel with the MMU enabled, so that the kernel might end up running with tcr_el1.t0sz value set-up for the identity mapping which can be lower than the value required by the actual virtual address space, resulting in an erroneous set-up.Quoted 4 lines
This patchs adds code in the resume path that restores the tcr_el1.t0sz default value upon core resume, mirroring this way the cold boot path behaviour therefore fixing the issue.
Quoted 6 lines
Of course, I agree with the patch, but could you condense the commit log, please? I don't think we need five paragraphs to point out that the sequence to set T0SZ to its idmap value, enable the MMU, branch into the kernel virtual mapping and set T0SZ to its default value is missing the final step on the resume path, and it actually took me a while to figure out what the problem was from reading the text.
Agreed.
Quoted 3 lines
Quoted 1 line
Fixes 95322526ef62 ("arm64: kernel: cpu_{suspend/resume} implementation")Wrong commit id
Technically speaking I am not fixing
Commit dd006da21646 ("arm64: mm: increase VA range of identity map")
I am fixing the commit I reported, because it never worked on platforms requiring extended idmap, if anything, your commit dd006da21646 gave CPUidle states and suspend2RAM a chance to work on those platforms, and this patch completes the implementation.
To make things simpler I can report:
as long as we all agree on the above.
Quoted 6 lines
Quoted 4 lines
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Will Deacon <[email protected]>Cc Catalin Marinas <[email protected]>Acked-by Ard Biesheuvel <[email protected]>
Thanks, I will prepare a v2 and ask Catalin to merge it at -rc1, it conflicts with patches currently on arm64 for-next/core we can wait till -rc1.
Thanks,
Lorenzo
PATCH 1/1 arm64: kernel: fix tcr_el1.t0sz restore on systems with extended idmap
Quoted 10 lines · Lorenzo Pieralisi
Hi Ard,
Quoted 6 lines · Ard Biesheuvel
Hi Lorenzo,
Thanks for the fix. I'm a bit embarrassed since this is quite obviously broken, and I (nor anyone else, for that matter) spotted it through testing suspend/resume on Seattle (frankly, I don't even know if suspend/resume works currently on Seattle or if it did at the time)
Quoted 5 lines
You could not spot it by testing on Seattle, it does not implement PM yet, I spotted it by code inspection while reviewing James' suspend-to-disk patches, there is nothing to be embarassed about, you paved the way to fix it.
Ah ok, that is good to hear.
Quoted 19 lines
Quoted 19 lines · Lorenzo Pieralisi
Quoted 17 lines
Commit
dd006da21646("arm64: mm: increase VA range of identity map") introduced a mechanism to extend the virtual memory map range to support arm64 systems with system RAM located at very high offset, where the identity mapping used to enable/disable the MMU requires additional translation levels to map the physical memory at an equal virtual offset.The kernel detects at boot time the tcr_el1.t0sz value required by the identity mapping and sets-up the tcr_el1.t0sz register field accordingly, any time the identity map is required in the kernel (ie when enabling the MMU).
After enabling the MMU, in the cold boot path the kernel resets the tcr_el1.t0sz to its default value (ie the actual configuration value for the system virtual address space) so that after enabling the MMU the memory space translated by
ttbr0_el1is restored as expected.
Quoted 6 lines
Quoted 2 lines
'After enabling the MMU' is confusing, especially since you use it twice to describe two different points in time.
I will sum up this commit log, it is ways too long and not clear at times.
OK
Quoted 37 lines
Quoted 15 lines
Quoted 10 lines
Commit
dd006da21646("arm64: mm: increase VA range of identity map") also added code to set-up the tcr_el1.t0sz value when the kernel resumes from low-power states with the MMU off throughcpu_resume()in order to effectively use the identity mapping to enable the MMU but failed to add the code required to restore the tcr_el1.t0sz to its default value, when the core returns to the kernel with the MMU enabled, so that the kernel might end up running with tcr_el1.t0sz value set-up for the identity mapping which can be lower than the value required by the actual virtual address space, resulting in an erroneous set-up.Quoted 4 lines
This patchs adds code in the resume path that restores the tcr_el1.t0sz default value upon core resume, mirroring this way the cold boot path behaviour therefore fixing the issue.
Quoted 6 lines
Of course, I agree with the patch, but could you condense the commit log, please? I don't think we need five paragraphs to point out that the sequence to set T0SZ to its idmap value, enable the MMU, branch into the kernel virtual mapping and set T0SZ to its default value is missing the final step on the resume path, and it actually took me a while to figure out what the problem was from reading the text.
Agreed.
Quoted 3 lines
Quoted 1 line
Fixes 95322526ef62 ("arm64: kernel: cpu_{suspend/resume} implementation")Wrong commit id
Technically speaking I am not fixing
Commit
dd006da21646("arm64: mm: increase VA range of identity map")I am fixing the commit I reported, because it never worked on platforms requiring extended idmap, if anything, your commit
dd006da21646gave CPUidle states and suspend2RAM a chance to work on those platforms, and this patch completes the implementation.
Quoted 6 lines
To make things simpler I can report:
Fixes dd006da21646 ("arm64: mm: increase VA range of identity map")as long as we all agree on the above.
This patch cannot be applied to any stable tree that does not have dd006da21646, since that commit introduces cpu_set_default_tcr_t0sz(), and your patch introduces a call to that function. However, the context does not reflect that, and the patch may apply cleanly to older kernels only to break at build time. So to make the lives of stable kernel maintainers easier, it is better to use the latter IMO.
Quoted 11 lines
Quoted 6 lines
Quoted 4 lines
Signed-off-by Lorenzo Pieralisi <[email protected]>Cc Ard Biesheuvel <[email protected]>Cc Will Deacon <[email protected]>Cc Catalin Marinas <[email protected]>Acked-by Ard Biesheuvel <[email protected]>Thanks, I will prepare a v2 and ask Catalin to merge it at -rc1, it conflicts with patches currently on arm64 for-next/core we can wait till -rc1.
OK.
Signature · 1 line
Ard.