arm64: kernel: avoid literal load of virtual address with MMU off
PATCH 1/1 arm64: kernel: avoid literal load of virtual address with MMU off
Literal loads of virtual addresses are subject to runtime relocation when CONFIG_RELOCATABLE=y, and given that the relocation routines run with the MMU and caches enabled, literal loads of relocated values performed with the MMU off are not guaranteed to return the latest value unless the memory covering the literal is cleaned to the PoC explicitly.
So defer the literal load until after the MMU has been enabled, just like we do for primary_switch() and secondary_switch() in head.S.
---
This conflicts with the x25/x26 patch I sent yesterday, but this should probably go into stable, so I based it on v4.8-rc directly.
arch/arm64/kernel/sleep.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
arch/arm64/kernel/sleep.S +9 -1
@@ -101,12 +101,20 @@ ENTRY(cpu_resume) bl el2_setup // if in EL2 drop to EL1 cleanly /* enable the MMU early - so we can access sleep_save_stash by va */ adr_l lr, __enable_mmu /* __cpu_setup will return here */ - ldr x27, =_cpu_resume /* __enable_mmu will branch here */ + adr_l x27, _resume_switched /* __enable_mmu will branch here */ adrp x25, idmap_pg_dir adrp x26, swapper_pg_dir b __cpu_setup ENDPROC(cpu_resume) + .pushsection ".idmap.text", "ax" +_resume_switched: + ldr x8, =_cpu_resume + br x8 +ENDPROC(_resume_switched) + .ltorg + .popsection + ENTRY(_cpu_resume) mrs x1, mpidr_el1 adrp x8, mpidr_hash
Signature · 1 line
2.7.4
Mark Rutland
reviewerPATCH 1/1 arm64: kernel: avoid literal load of virtual address with MMU off
Quoted 11 lines · Ard Biesheuvel
Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the MMU and caches enabled, literal loads of relocated values performed with the MMU off are not guaranteed to return the latest value unless the memory covering the literal is cleaned to the PoC explicitly.So defer the literal load until after the MMU has been enabled, just like we do for
primary_switch()andsecondary_switch()inhead.S.Fixes 1e48ef7fcc37 ("arm64: add support for building vmlinux as a relocatable PIE binary")Signed-off-by Ard Biesheuvel <[email protected]>
This looks like the simplest way to handle this, and is consistent with what we do elsewhere, so FWIW:
Quoted 1 line
From grepping, this seems to be the only case of a relocated literal
being loaded while the MMU is off under arch/arm64/.
Thanks,
Mark.
Quoted 37 lines
---
This conflicts with the x25/x26 patch I sent yesterday, but this should probably go into stable, so I based it on v4.8-rc directly.
arch/arm64/kernel/sleep.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)arch/arm64/kernel/sleep.S +9
-1@@ -101,12 +101,20 @@ ENTRY(cpu_resume) bl el2_setup // if in EL2 drop to EL1 cleanly /* enable the MMU early - so we can access sleep_save_stash by va */ adr_l lr, __enable_mmu /* __cpu_setup will return here */ - ldr x27, =_cpu_resume /* __enable_mmu will branch here */ + adr_l x27, _resume_switched /* __enable_mmu will branch here */ adrp x25, idmap_pg_dir adrp x26, swapper_pg_dir b __cpu_setup ENDPROC(cpu_resume) + .pushsection ".idmap.text", "ax" +_resume_switched: + ldr x8, =_cpu_resume + br x8 +ENDPROC(_resume_switched) + .ltorg + .popsection + ENTRY(_cpu_resume) mrs x1, mpidr_el1 adrp x8, mpidr_hashSignature · 1 line
2.7.4
Catalin Marinas
maintainerPATCH 1/1 arm64: kernel: avoid literal load of virtual address with MMU off
In reply to Ard BiesheuvelQuoted 8 lines · Ard Biesheuvel
Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the MMU and caches enabled, literal loads of relocated values performed with the MMU off are not guaranteed to return the latest value unless the memory covering the literal is cleaned to the PoC explicitly.So defer the literal load until after the MMU has been enabled, just like we do for
primary_switch()andsecondary_switch()inhead.S.
I think for consistency with head.S the patch is fine but do we actually expect a literal pool value to be inconsistent between cache and PoC? Presumably, the first time the kernel image was loaded it had to be cleaned to PoC but after that we wouldn't expect modifications of the literal pool data.
Signature · 1 line
Catalin
Mark Rutland
reviewerPATCH 1/1 arm64: kernel: avoid literal load of virtual address with MMU off
Quoted 7 lines · Catalin Marinas
Quoted 6 lines · Ard Biesheuvel
Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the MMU and caches enabled, literal loads of relocated values performed with the MMU off are not guaranteed to return the latest value unless the memory covering the literal is cleaned to the PoC explicitly.
Quoted 8 lines
Quoted 2 lines
So defer the literal load until after the MMU has been enabled, just like we do for
primary_switch()andsecondary_switch()inhead.S.I think for consistency with
head.Sthe patch is fine but do we actually expect a literal pool value to be inconsistent between cache and PoC? Presumably, the first time the kernel image was loaded it had to be cleaned to PoC but after that we wouldn't expect modifications of the literal pool data.
The kernel modifies literal pool values within itself, at boot time, after having enabled the MMU. This is after any loader has cleaned the kernel Image to the PoC and branched to it with the MMU off.
See __primary_switch in head.S
With upcoming patches [1], resuming from hibernation can also leave stale data at the PoC for relocated literals like this.
Thanks,
Mark.
[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2016-August/448996.html
PATCH 1/1 arm64: kernel: avoid literal load of virtual address with MMU off
In reply to Catalin MarinasQuoted 7 lines · Catalin Marinas
Quoted 6 lines · Ard Biesheuvel
Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the MMU and caches enabled, literal loads of relocated values performed with the MMU off are not guaranteed to return the latest value unless the memory covering the literal is cleaned to the PoC explicitly.
Quoted 5 lines
Quoted 2 lines
So defer the literal load until after the MMU has been enabled, just like we do for
primary_switch()andsecondary_switch()inhead.S.I think for consistency with
head.Sthe patch is fine but do we actually expect a literal pool value to be inconsistent between cache and PoC?
Yes. Any literals covered by R_AARCH64_ABS64 relocations at link time will be covered by R_AARCH64_RELATIVE at runtime (when CONFIG_RELOCATABLE=y). Since AArch64 uses the RELA format, the targets of the relocations will contain zeroes before the relocation routine runs, and no cache maintenance is performed. This means a relocatable image requires runtime processing even if it is loaded unrandomised.
Quoted 6 lines
Presumably, the first time the kernel image was loaded it had to be cleaned to PoC but after that we wouldn't expect modifications of the literal pool dataz
--
Catalin
Catalin Marinas
maintainerPATCH 1/1 arm64: kernel: avoid literal load of virtual address with MMU off
Quoted 14 lines · Ard Biesheuvel
Quoted 7 lines · Catalin Marinas
Quoted 6 lines · Ard Biesheuvel
Literal loads of virtual addresses are subject to runtime relocation when
CONFIG_RELOCATABLE=y, and given that the relocation routines run with the MMU and caches enabled, literal loads of relocated values performed with the MMU off are not guaranteed to return the latest value unless the memory covering the literal is cleaned to the PoC explicitly.Quoted 5 lines
Quoted 2 lines
So defer the literal load until after the MMU has been enabled, just like we do for
primary_switch()andsecondary_switch()inhead.S.I think for consistency with
head.Sthe patch is fine but do we actually expect a literal pool value to be inconsistent between cache and PoC?
Quoted 6 lines
Yes. Any literals covered by
R_AARCH64_ABS64relocations at link time will be covered byR_AARCH64_RELATIVEat runtime (whenCONFIG_RELOCATABLE=y). Since AArch64 uses the RELA format, the targets of the relocations will contain zeroes before the relocation routine runs, and no cache maintenance is performed. This means a relocatable image requires runtime processing even if it is loaded unrandomised.
Ah, I forgot about this. Patch queued for 4.8-rc3. Thanks.
Signature · 1 line
Catalin