Skip to contentSkip to conversation

Fix ftracetest issues

PATCH New Russell King - ARM Linux 10 participants 21 messages Nov 25, 2017 – Dec 22, 2017

Subsystems (3)
lore
Share & tools
lore

Russell King - ARM Linux

author maintainer
#0 lore raw

cover Fix ftracetest issues

ftracetest provokes the kernel to try and return to userspace addresses in kernel mode. These two patches prevent that.

The first patch, which I intend merging with Linus tonight, ensures that we catch the condition before we hit userspace, meaning that there is no possibility of executing user code while in kernel mode.

The second patch fixes the ftracetest issue itself by ensuring that it is not possible to set a kprobe on any of the "special" assembler code. Such code includes:
- the kernel primary/secondary CPU startup code
- exception entry code
- idmap code

This is because the conditions under which this code is executed does not meet the kprobes requirements, which is basically that the "function" is C-like - it does something and then returns to the parent, and has a stack. This is just not universally true of the above code.

This patch is larger than one may desire as we re-organise the sections that some code ends up in, the way the unwinder works, and how we print stack frames.

 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/entry-header.S   |  6 ++++++
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 12 files changed, 68 insertions(+), 42 deletions(-)
Signature · 3 lines
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
#1 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

---
 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/kernel/entry-header.S   |  6 ++++++
 2 files changed, 24 insertions(+)
arch/arm/include/asm/assembler.h +18 -0
index ad301f107dd2..bc8d4bbd82e2 100644
@@ -518,4 +518,22 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 #endif
 	.endm
 
+	.macro	bug, msg, line
+#ifdef CONFIG_THUMB2_KERNEL
+1:	.inst	0xde02
+#else
+1:	.inst	0xe7f001f2
+#endif
+#ifdef CONFIG_DEBUG_BUGVERBOSE
+	.pushsection .rodata.str, "aMS", %progbits, 1
+2:	.asciz	"\msg"
+	.popsection
+	.pushsection __bug_table, "aw"
+	.align	2
+	.word	1b, 2b
+	.hword	\line
+	.popsection
+#endif
+	.endm
+
 #endif /* __ASM_ASSEMBLER_H__ */
arch/arm/kernel/entry-header.S +6 -0
index d523cd8439a3..7f4d80c2db6b 100644
@@ -300,6 +300,8 @@
 	mov	r2, sp
 	ldr	r1, [r2, #\offset + S_PSR]	@ get calling cpsr
 	ldr	lr, [r2, #\offset + S_PC]!	@ get pc
+	tst	r1, #0xcf
+	bne	1f
 	msr	spsr_cxsf, r1			@ save in spsr_svc
 #if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K)
 	@ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -314,6 +316,7 @@
 						@ after ldm {}^
 	add	sp, sp, #\offset + PT_REGS_SIZE
 	movs	pc, lr				@ return & move spsr_svc into cpsr
+1:	bug	"Returning to usermode but unexpected PSR bits set?", \@
 #elif defined(CONFIG_CPU_V7M)
 	@ V7M restore.
 	@ Note that we don't need to do clrex here as clearing the local
@@ -329,6 +332,8 @@
 	ldr	r1, [sp, #\offset + S_PSR]	@ get calling cpsr
 	ldr	lr, [sp, #\offset + S_PC]	@ get pc
 	add	sp, sp, #\offset + S_SP
+	tst	r1, #0xcf
+	bne	1f
 	msr	spsr_cxsf, r1			@ save in spsr_svc
 
 	@ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -341,6 +346,7 @@
 	.endif
 	add	sp, sp, #PT_REGS_SIZE - S_SP
 	movs	pc, lr				@ return & move spsr_svc into cpsr
+1:	bug	"Returning to usermode but unexpected PSR bits set?", \@
 #endif	/* !CONFIG_THUMB2_KERNEL */
 	.endm
 
Signature · 1 line
2.7.4
#2 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Hello Russell,

Quoted 9 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

---

This patch breaks my 32 bit ARM system when running under Qemu. I get this continually:

[    2.130043] ------------[ cut here ]------------
[    2.130132] kernel BUG at Returning to usermode but unexpected PSR
bits set?:9!
[    2.130233] Internal error: Oops - BUG: 0 [#1] ARM
[    2.130375] Modules linked in:
[    2.130805] CPU: 0 PID: 154 Comm: modprobe Not tainted 4.15.0-rc1 #3
[    2.130874] Hardware name: Generic DT based system
[    2.131023] task: 87a02800 task.stack: 87970000
[    2.131158] PC is at no_work_pending+0x2c/0x30
[    2.131402] LR is at 0x76f18ae8
[    2.131462] pc : [<8000a600>]    lr : [<76f18ae8>]    psr: 200001d3
[    2.131516] sp : 87971fb0  ip : 80014484  fp : 00000000
[    2.131567] r10: 00000000  r9 : 87970000  r8 : 00000000
[    2.131627] r7 : 00c5387d  r6 : ffffffff  r5 : 00000150  r4 : 76f18ae8
[    2.131686] r3 : 00000000  r2 : 87971fec  r1 : 00000150  r0 : 00000000
[    2.131818] Flags: nzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM
Segment user
[    2.131894] Control: 00c5387d  Table: 8794c008  DAC: 00000055
[    2.131971] Process modprobe (pid: 154, stack limit = 0x87970188)
[    2.132075] Stack: (0x87971fb0 to 0x87972000)
[    2.132273] 1fa0:                                     00000000
00000000 00000000 00000000
[    2.132344] 1fc0: 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000
[    2.132395] 1fe0: 00000000 7ec5fec0 00000000 76f18ae8 00000150
ffffffff e3a00001 e58d300c
[    2.133146] Code: e9527fff e1a00000 e28dd048 e1b0f00e (e7f001f2)
[    2.133593] ---[ end trace 46087be8f22855bc ]---

This is 4.15-rc1, booting aspeed_g5_defconfig on Qemu master (v2.11.0-rc2-14-ge7b47c2).

qemu-system-arm -nographic -nodefaults -serial stdio -M romulus-bmc \
-kernel arch/arm/boot/zImage -dtb
arch/arm/boot/dts/aspeed-bmc-opp-romulus.dtb \
-initrd arm.cpio.xz

If I revert the patch userspace runs as expected.

Cheers,

Joel

Quoted 73 lines
 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/kernel/entry-header.S   |  6 ++++++
 2 files changed, 24 insertions(+)
arch/arm/include/asm/assembler.h +18 -0
index ad301f107dd2..bc8d4bbd82e2 100644
@@ -518,4 +518,22 @@ THUMB(     orr     \reg , \reg , #PSR_T_BIT        )
 #endif
        .endm
 
+       .macro  bug, msg, line
+#ifdef CONFIG_THUMB2_KERNEL
+1:     .inst   0xde02
+#else
+1:     .inst   0xe7f001f2
+#endif
+#ifdef CONFIG_DEBUG_BUGVERBOSE
+       .pushsection .rodata.str, "aMS", %progbits, 1
+2:     .asciz  "\msg"
+       .popsection
+       .pushsection __bug_table, "aw"
+       .align  2
+       .word   1b, 2b
+       .hword  \line
+       .popsection
+#endif
+       .endm
+
 #endif /* __ASM_ASSEMBLER_H__ */
arch/arm/kernel/entry-header.S +6 -1
index d523cd8439a3..7f4d80c2db6b 100644
@@ -300,6 +300,8 @@
        mov     r2, sp
        ldr     r1, [r2, #\offset + S_PSR]      @ get calling cpsr
        ldr     lr, [r2, #\offset + S_PC]!      @ get pc
+       tst     r1, #0xcf
+       bne     1f
        msr     spsr_cxsf, r1                   @ save in spsr_svc
 #if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K)
        @ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -314,6 +316,7 @@
                                                @ after ldm {}^
        add     sp, sp, #\offset + PT_REGS_SIZE
        movs    pc, lr                          @ return & move spsr_svc into cpsr
+1:     bug     "Returning to usermode but unexpected PSR bits set?", \@
 #elif defined(CONFIG_CPU_V7M)
        @ V7M restore.
        @ Note that we don't need to do clrex here as clearing the local
@@ -329,6 +332,8 @@
        ldr     r1, [sp, #\offset + S_PSR]      @ get calling cpsr
        ldr     lr, [sp, #\offset + S_PC]       @ get pc
        add     sp, sp, #\offset + S_SP
+       tst     r1, #0xcf
+       bne     1f
        msr     spsr_cxsf, r1                   @ save in spsr_svc
 
        @ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -341,6 +346,7 @@
        .endif
        add     sp, sp, #PT_REGS_SIZE - S_SP
        movs    pc, lr                          @ return & move spsr_svc into cpsr
+1:     bug     "Returning to usermode but unexpected PSR bits set?", \@
 #endif /* !CONFIG_THUMB2_KERNEL */
        .endm
 
--

2.7.4

Quoted 4 lines

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: linux-4.15-rc1-dmesg
Type: application/octet-stream
Size: 11762 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20171127/44b0b540/attachment-0001.obj>

author maintainer
#3 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 37 lines · Joel Stanley

Hello Russell,

Quoted 9 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

---

This patch breaks my 32 bit ARM system when running under Qemu. I get this continually:

[    2.130043] ------------[ cut here ]------------
[    2.130132] kernel BUG at Returning to usermode but unexpected PSR
bits set?:9!
[    2.130233] Internal error: Oops - BUG: 0 [#1] ARM
[    2.130375] Modules linked in:
[    2.130805] CPU: 0 PID: 154 Comm: modprobe Not tainted 4.15.0-rc1 #3
[    2.130874] Hardware name: Generic DT based system
[    2.131023] task: 87a02800 task.stack: 87970000
[    2.131158] PC is at no_work_pending+0x2c/0x30
[    2.131402] LR is at 0x76f18ae8
[    2.131462] pc : [<8000a600>]    lr : [<76f18ae8>]    psr: 200001d3
[    2.131516] sp : 87971fb0  ip : 80014484  fp : 00000000
[    2.131567] r10: 00000000  r9 : 87970000  r8 : 00000000
[    2.131627] r7 : 00c5387d  r6 : ffffffff  r5 : 00000150  r4 : 76f18ae8
[    2.131686] r3 : 00000000  r2 : 87971fec  r1 : 00000150  r0 : 00000000
[    2.131818] Flags: nzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM
Segment user
[    2.131894] Control: 00c5387d  Table: 8794c008  DAC: 00000055
[    2.131971] Process modprobe (pid: 154, stack limit = 0x87970188)
[    2.132075] Stack: (0x87971fb0 to 0x87972000)
Quoted 8 lines
[    2.132273] 1fa0:                                     00000000
00000000 00000000 00000000
[    2.132344] 1fc0: 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000
[    2.132395] 1fe0: 00000000 7ec5fec0 00000000 76f18ae8 00000150
ffffffff e3a00001 e58d300c
[    2.133146] Code: e9527fff e1a00000 e28dd048 e1b0f00e (e7f001f2)
[    2.133593] ---[ end trace 46087be8f22855bc ]---

It looks like FIQs are disabled when returning to userspace. Also it looks like imprecise aborts were disabled too, which isn't good for running userspace.

As we explicitly set the userspace CPSR value when we exec a program, these bits should not be set. The mostly-zeros dump of the registers in the stack with the exception of old_r0 being ~0 suggests that we were are handling an exception very close to the start of execution of modprobe - maybe a prefetch abort to fault the first page of executable code in.

This has the feeling of being a qemu bug.

We could reduce the mask being tested in the patch to exclude the FIQ bit. However, I'd like to see it investigated further - but qemu and myself do not get along - my only success with it was running some MIPS code. I've never had it do anything useful with ARM stuff.

Signature · 3 lines
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
#4 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 51 lines · Russell King - ARM Linux
Quoted 37 lines · Joel Stanley

Hello Russell,

Quoted 9 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

---

This patch breaks my 32 bit ARM system when running under Qemu. I get this continually:

[    2.130043] ------------[ cut here ]------------
[    2.130132] kernel BUG at Returning to usermode but unexpected PSR
bits set?:9!
[    2.130233] Internal error: Oops - BUG: 0 [#1] ARM
[    2.130375] Modules linked in:
[    2.130805] CPU: 0 PID: 154 Comm: modprobe Not tainted 4.15.0-rc1 #3
[    2.130874] Hardware name: Generic DT based system
[    2.131023] task: 87a02800 task.stack: 87970000
[    2.131158] PC is at no_work_pending+0x2c/0x30
[    2.131402] LR is at 0x76f18ae8
[    2.131462] pc : [<8000a600>]    lr : [<76f18ae8>]    psr: 200001d3
[    2.131516] sp : 87971fb0  ip : 80014484  fp : 00000000
[    2.131567] r10: 00000000  r9 : 87970000  r8 : 00000000
[    2.131627] r7 : 00c5387d  r6 : ffffffff  r5 : 00000150  r4 : 76f18ae8
[    2.131686] r3 : 00000000  r2 : 87971fec  r1 : 00000150  r0 : 00000000
[    2.131818] Flags: nzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM
Segment user
[    2.131894] Control: 00c5387d  Table: 8794c008  DAC: 00000055
[    2.131971] Process modprobe (pid: 154, stack limit = 0x87970188)
[    2.132075] Stack: (0x87971fb0 to 0x87972000)
Quoted 8 lines
[    2.132273] 1fa0:                                     00000000
00000000 00000000 00000000
[    2.132344] 1fc0: 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000
[    2.132395] 1fe0: 00000000 7ec5fec0 00000000 76f18ae8 00000150
ffffffff e3a00001 e58d300c
[    2.133146] Code: e9527fff e1a00000 e28dd048 e1b0f00e (e7f001f2)
[    2.133593] ---[ end trace 46087be8f22855bc ]---

It looks like FIQs are disabled when returning to userspace. Also it looks like imprecise aborts were disabled too, which isn't good for running userspace.

Quoted 8 lines

As we explicitly set the userspace CPSR value when we exec a program, these bits should not be set. The mostly-zeros dump of the registers in the stack with the exception of old_r0 being ~0 suggests that we were are handling an exception very close to the start of execution of modprobe - maybe a prefetch abort to fault the first page of executable code in.

This has the feeling of being a qemu bug.

Additional QEMU logging gives us :

Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear

It has been the case for a while but the kernel did not panic before. When U-Boot loads the kernel, the problem does not show up. I wonder which setting it's doing.

C.

Quoted 5 lines

We could reduce the mask being tested in the patch to exclude the FIQ bit. However, I'd like to see it investigated further - but qemu and myself do not get along - my only success with it was running some MIPS code. I've never had it do anything useful with ARM stuff.

author maintainer
#5 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 59 lines · C?dric Le Goater
Quoted 58 lines · Russell King - ARM Linux
Quoted 37 lines · Joel Stanley

Hello Russell,

Quoted 9 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

---

This patch breaks my 32 bit ARM system when running under Qemu. I get this continually:

[    2.130043] ------------[ cut here ]------------
[    2.130132] kernel BUG at Returning to usermode but unexpected PSR
bits set?:9!
[    2.130233] Internal error: Oops - BUG: 0 [#1] ARM
[    2.130375] Modules linked in:
[    2.130805] CPU: 0 PID: 154 Comm: modprobe Not tainted 4.15.0-rc1 #3
[    2.130874] Hardware name: Generic DT based system
[    2.131023] task: 87a02800 task.stack: 87970000
[    2.131158] PC is at no_work_pending+0x2c/0x30
[    2.131402] LR is at 0x76f18ae8
[    2.131462] pc : [<8000a600>]    lr : [<76f18ae8>]    psr: 200001d3
[    2.131516] sp : 87971fb0  ip : 80014484  fp : 00000000
[    2.131567] r10: 00000000  r9 : 87970000  r8 : 00000000
[    2.131627] r7 : 00c5387d  r6 : ffffffff  r5 : 00000150  r4 : 76f18ae8
[    2.131686] r3 : 00000000  r2 : 87971fec  r1 : 00000150  r0 : 00000000
[    2.131818] Flags: nzCv  IRQs off  FIQs off  Mode SVC_32  ISA ARM
Segment user
[    2.131894] Control: 00c5387d  Table: 8794c008  DAC: 00000055
[    2.131971] Process modprobe (pid: 154, stack limit = 0x87970188)
[    2.132075] Stack: (0x87971fb0 to 0x87972000)
Quoted 8 lines
[    2.132273] 1fa0:                                     00000000
00000000 00000000 00000000
[    2.132344] 1fc0: 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000
[    2.132395] 1fe0: 00000000 7ec5fec0 00000000 76f18ae8 00000150
ffffffff e3a00001 e58d300c
[    2.133146] Code: e9527fff e1a00000 e28dd048 e1b0f00e (e7f001f2)
[    2.133593] ---[ end trace 46087be8f22855bc ]---

It looks like FIQs are disabled when returning to userspace. Also it looks like imprecise aborts were disabled too, which isn't good for running userspace.

As we explicitly set the userspace CPSR value when we exec a program, these bits should not be set. The mostly-zeros dump of the registers in the stack with the exception of old_r0 being ~0 suggests that we were are handling an exception very close to the start of execution of modprobe - maybe a prefetch abort to fault the first page of executable code in.

Quoted 6 lines
Quoted 1 line

This has the feeling of being a qemu bug.

Additional QEMU logging gives us :

Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear

Thanks for finding that.

Quoted 1 line

It has been the case for a while but the kernel did not panic before.

That's because the patch that I added over the weekend wasn't in the kernel before - and the patch is there to prevent any possibility of user code being executed in SVC mode by the kernel.

The reason for rushing that in was because there's a proven path where it's possible for the kernel to do _exactly_ that, and if it's possible to run user code in SVC mode, then the security of the system has been compromised.

Given the serious nature of that, I'm not about to consider reverting the patch on the grounds that a software emulation doesn't like it - but I will drop the FIQ bit from the check which should avoid the issue on qemu.

Quoted 4 lines

When U-Boot loads the kernel, the problem does not show up. I wonder which setting it's doing.

From what it sounds like, when you run the kernel without uboot, QEMU

decides to run the kernel in the nonsecure world. If it did the same with u-boot, u-boot would not be able to change the SCR register, so u-boot must run in the secure world if it's setting the AW and FW bits.

It's correct for uboot to come up with the SCR as zero, because that's what a real CPU would do.

In any case, this behaviour is something to be aware of when running the kernel in qemu - it means that it's *not* going to behave the same way under qemu as on real hardware because imprecise aborts are masked and the kernel is unable to unmask them. On real hardware, those aborts will be unmasked. I'd suggest that qemu should make that blindingly obvious (eg, a message at startup if it's being asked to run the kernel in non-secure mode with at least the SCR.AW bit clear.)

Signature · 3 lines
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
#6 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 11 lines · Russell King - ARM Linux
Quoted 4 lines · C?dric Le Goater

Additional QEMU logging gives us :

Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear

From what it sounds like, when you run the kernel without uboot, QEMU decides to run the kernel in the nonsecure world. If it did the same with u-boot, u-boot would not be able to change the SCR register, so u-boot must run in the secure world if it's setting the AW and FW bits.

Yes, this seems like a QEMU bug. (Added qemu-devel to cc: accordingly). We'll print this warning for a v7 CPU which has the Security Extensions but not the Virtualization Extensions (which is the case where the architecture mandates that NS not be able to change the CPSR.A and F bits), but we shouldn't (in theory) get into that situation because either:
* we started the emulation by running a secure bootrom, which should have initialized SCR.AW/FW correctly before booting the NS kernel (I assume this is the "run from u-boot" setup)
* we started the kernel directly from QEMU, in which case QEMU's internal "act like a bootloader" code should have done the setting of SCR.AW/FW.

Cedric's command line is the second of these types, so something's gone wrong in our internal bootloader. The code in hw/arm/boot.c:arm_write_secure_board_setup_dummy_smc() sets up a little bit of guest code that runs in secure mode and sets SCR.AW/FW. I suspect the romulus-bmc machine model isn't invoking that code.

Most QEMU users shouldn't be affected by this because not many board models default to enabling the security extensions in the emulated CPU, and the highbank and raspi boards at least do do this secure-mode setup I think.

thanks
-- PMM

#7 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 27 lines · Peter Maydell
Quoted 11 lines · Russell King - ARM Linux
Quoted 4 lines · C?dric Le Goater

Additional QEMU logging gives us :

Ignoring attempt to switch CPSR_A flag from non-secure world with SCR.AW bit clear Ignoring attempt to switch CPSR_F flag from non-secure world with SCR.FW bit clear

From what it sounds like, when you run the kernel without uboot, QEMU decides to run the kernel in the nonsecure world. If it did the same with u-boot, u-boot would not be able to change the SCR register, so u-boot must run in the secure world if it's setting the AW and FW bits.

Yes, this seems like a QEMU bug. (Added qemu-devel to cc: accordingly). We'll print this warning for a v7 CPU which has the Security Extensions but not the Virtualization Extensions (which is the case where the architecture mandates that NS not be able to change the CPSR.A and F bits), but we shouldn't (in theory) get into that situation because either:
* we started the emulation by running a secure bootrom, which should have initialized SCR.AW/FW correctly before booting the NS kernel (I assume this is the "run from u-boot" setup)
* we started the kernel directly from QEMU, in which case QEMU's internal "act like a bootloader" code should have done the setting of SCR.AW/FW.

Quoted 6 lines

Cedric's command line is the second of these types, so something's gone wrong in our internal bootloader. The code in hw/arm/boot.c:arm_write_secure_board_setup_dummy_smc() sets up a little bit of guest code that runs in secure mode and sets SCR.AW/FW. I suspect the romulus-bmc machine model isn't invoking that code.

Indeed, it is missing.

With a quick hack adding the extra secure boot loader, I could boot the QEMU romulus-bmc machine (armv6 CPU) with 4.15-rc1 kernel. All is fine.

But the code of the Aspeed machines also covers the ast2400 SoC which uses an armv5 CPU. So it needs some rework. That will be for QEMU 2.12 I suppose.

Thank,

C.

Quoted 8 lines

Most QEMU users shouldn't be affected by this because not many board models default to enabling the security extensions in the emulated CPU, and the highbank and raspi boards at least do do this secure-mode setup I think.

thanks
-- PMM

#8 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

In reply to Russell King

Dear All,

Quoted 8 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

This patch brakes kernel booting on Exynos 4412-based Odroid U3 board:

[??? 4.211412] ------------[ cut here ]------------
[??? 4.211553] kernel BUG at Returning to usermode but unexpected PSR bits set?:5!
[??? 4.217696] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM [??? 4.223513] Modules linked in:
[??? 4.226557] CPU: 3 PID: 1 Comm: init Not tainted 4.14.0-rc1-00013-g8bafae202c82 #3453
[??? 4.234362] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree) [??? 4.240440] task: ee870000 task.stack: ee86e000
[??? 4.244977] PC is at no_work_pending+0x38/0x3c
[??? 4.249380] LR is at 0xb6f73b00
[??? 4.252502] pc : [<c010880c>]??? lr : [<b6f73b00>] psr: 200001d3 [??? 4.258752] sp : ee86ffb0? ip : 00000530? fp : 00000000 [??? 4.263960] r10: 00000000? r9 : ee86e000? r8 : 00000000 [??? 4.269168] r7 : 10c5387d? r6 : ffffffff? r5 : 00000050? r4 : b6f73b00 [??? 4.275678] r3 : ee870000? r2 : ee86ffec? r1 : 00000050? r0 : c01087d8 [??? 4.282190] Flags: nzCv? IRQs off? FIQs off? Mode SVC_32? ISA ARM? Segment user
[??? 4.289478] Control: 10c5387d? Table: 6df9004a? DAC: 00000055 [??? 4.295208] Process init (pid: 1, stack limit = 0xee86e210) [??? 4.300763] Stack: (0xee86ffb0 to 0xee870000)
[??? 4.305104] ffa0:???????????????????????????????????? 00000000 00000000 00000000 00000000
[??? 4.313262] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[??? 4.321422] ffe0: 00000000 bed6ef00 00000000 b6f73b00 00000050 ffffffff 00000000 00000000
[??? 4.329585] Code: e9527fff e1a00000 e28dd048 e1b0f00e (e7f001f2) [??? 4.335663] ---[ end trace 8579492d646004d9 ]---
[??? 4.340263] BUG: sleeping function called from invalid context at ./include/linux/percpu-rwsem.h:33
[??? 4.349284] in_atomic(): 0, irqs_disabled(): 128, pid: 1, name: init [??? 4.355622] INFO: lockdep is turned off.
[??? 4.359525] irq event stamp: 258120
[??? 4.363011] hardirqs last? enabled at (258119): [<c01ba41c>] filemap_map_pages+0x380/0x540
[??? 4.371252] hardirqs last disabled at (258120): [<c010d640>] __und_svc+0x60/0x74
[??? 4.378627] softirqs last? enabled at (257808): [<c0101784>] __do_softirq+0x224/0x318
[??? 4.386443] softirqs last disabled at (257783): [<c01221a8>] irq_exit+0xe4/0x128
[??? 4.393816] CPU: 3 PID: 1 Comm: init Tainted: G????? D 4.14.0-rc1-00013-g8bafae202c82 #3453
[??? 4.402838] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree) [??? 4.408926] [<c0110054>] (unwind_backtrace) from [<c010c86c>] (show_stack+0x10/0x14)
[??? 4.416660] [<c010c86c>] (show_stack) from [<c08266e4>] (dump_stack+0x90/0xc8)
[??? 4.423858] [<c08266e4>] (dump_stack) from [<c0142b94>] (___might_sleep+0x22c/0x2c8)
[??? 4.431581] [<c0142b94>] (___might_sleep) from [<c012cb6c>] (exit_signals+0x20/0x2f0)
[??? 4.439387] [<c012cb6c>] (exit_signals) from [<c01208b8>] (do_exit+0xb4/0xa94)
[??? 4.446589] [<c01208b8>] (do_exit) from [<c010ca7c>] (die+0x20c/0x2e4) [??? 4.453097] [<c010ca7c>] (die) from [<c01010c4>] (do_undefinstr+0xc0/0x270)
[??? 4.460042] [<c01010c4>] (do_undefinstr) from [<c010d65c>] (__und_svc_finish+0x0/0x44)
[??? 4.467939] Exception stack(0xee86ff60 to 0xee86ffa8)
[??? 4.472974] ff60: c01087d8 00000050 ee86ffec ee870000 b6f73b00 00000050 ffffffff 10c5387d
[??? 4.481134] ff80: 00000000 ee86e000 00000000 00000000 00000530 ee86ffb0 b6f73b00 c010880c
[??? 4.489291] ffa0: 200001d3 ffffffff
[??? 4.492769] [<c010d65c>] (__und_svc_finish) from [<c010880c>] (no_work_pending+0x38/0x3c)
[??? 4.525364] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[??? 4.525364]
[??? 4.528951] CPU1: stopping
[??? 4.531612] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G????? D W?????? 4.14.0-rc1-00013-g8bafae202c82 #3453
[??? 4.541050] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree) [??? 4.547181] [<c0110054>] (unwind_backtrace) from [<c010c86c>] (show_stack+0x10/0x14)
[??? 4.554895] [<c010c86c>] (show_stack) from [<c08266e4>] (dump_stack+0x90/0xc8)
[??? 4.562086] [<c08266e4>] (dump_stack) from [<c010f0bc>] (handle_IPI+0x174/0x1ac)
[??? 4.569464] [<c010f0bc>] (handle_IPI) from [<c0101554>] (gic_handle_irq+0x94/0xa0)
[??? 4.577009] [<c0101554>] (gic_handle_irq) from [<c010d570>] (__irq_svc+0x70/0xb0)
[??? 4.584454] Exception stack(0xee8c1f78 to 0xee8c1fc0)
[??? 4.589494] 1f60:?????????????????????????????????????????????????????? c01092e8 00000000
[??? 4.597664] 1f80: ee8c1fc8 00000000 c0d07c7c c0d07c20 c0c687b0 c0d07c88 c0d5e121 00000000
[??? 4.605821] 1fa0: 00000000 00000000 00000001 ee8c1fc8 c01092e8 c01092ec 60000053 ffffffff
[??? 4.614000] [<c010d570>] (__irq_svc) from [<c01092ec>] (arch_cpu_idle+0x20/0x3c)
[??? 4.621376] [<c01092ec>] (arch_cpu_idle) from [<c015dbe4>] (do_idle+0x144/0x214)
[??? 4.628744] [<c015dbe4>] (do_idle) from [<c015dff0>] (cpu_startup_entry+0x18/0x1c)
[??? 4.636294] [<c015dff0>] (cpu_startup_entry) from [<4010190c>] (0x4010190c)
[??? 4.643220] CPU0: stopping
[??? 4.645924] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G????? D W?????? 4.14.0-rc1-00013-g8bafae202c82 #3453
[??? 4.655365] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree) [??? 4.661486] [<c0110054>] (unwind_backtrace) from [<c010c86c>] (show_stack+0x10/0x14)
[??? 4.669207] [<c010c86c>] (show_stack) from [<c08266e4>] (dump_stack+0x90/0xc8)
[??? 4.676405] [<c08266e4>] (dump_stack) from [<c010f0bc>] (handle_IPI+0x174/0x1ac)
[??? 4.683776] [<c010f0bc>] (handle_IPI) from [<c0101554>] (gic_handle_irq+0x94/0xa0)
[??? 4.691322] [<c0101554>] (gic_handle_irq) from [<c010d570>] (__irq_svc+0x70/0xb0)
[??? 4.698768] Exception stack(0xc0d01f40 to 0xc0d01f88)
[??? 4.703823] 1f40: c01092e8 00000000 c0d01f90 00000000 c0d07c7c c0d07c20 c0c687b0 c0d07c88
[??? 4.711980] 1f60: c0d5e121 00000000 00000000 00000000 00000001 c0d01f90 c01092e8 c01092ec
[??? 4.720122] 1f80: 60000153 ffffffff
[??? 4.723625] [<c010d570>] (__irq_svc) from [<c01092ec>] (arch_cpu_idle+0x20/0x3c)
[??? 4.730998] [<c01092ec>] (arch_cpu_idle) from [<c015dbe4>] (do_idle+0x144/0x214)
[??? 4.738367] [<c015dbe4>] (do_idle) from [<c015dff0>] (cpu_startup_entry+0x18/0x1c)
[??? 4.745949] [<c015dff0>] (cpu_startup_entry) from [<c0c00c40>] (start_kernel+0x354/0x3bc)
[??? 4.754065] CPU2: stopping
[??? 4.756768] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G????? D W?????? 4.14.0-rc1-00013-g8bafae202c82 #3453
[??? 4.766208] Hardware name: SAMSUNG EXYNOS (Flattened Device Tree) [??? 4.772330] [<c0110054>] (unwind_backtrace) from [<c010c86c>] (show_stack+0x10/0x14)
[??? 4.780049] [<c010c86c>] (show_stack) from [<c08266e4>] (dump_stack+0x90/0xc8)
[??? 4.787247] [<c08266e4>] (dump_stack) from [<c010f0bc>] (handle_IPI+0x174/0x1ac)
[??? 4.794619] [<c010f0bc>] (handle_IPI) from [<c0101554>] (gic_handle_irq+0x94/0xa0)
[??? 4.802165] [<c0101554>] (gic_handle_irq) from [<c010d570>] (__irq_svc+0x70/0xb0)
[??? 4.809612] Exception stack(0xee8c3f78 to 0xee8c3fc0)
[??? 4.814651] 3f60:?????????????????????????????????????????????????????? c01092e8 00000000
[??? 4.822823] 3f80: ee8c3fc8 00000000 c0d07c7c c0d07c20 c0c687b0 c0d07c88 c0d5e121 00000000
[??? 4.830980] 3fa0: 00000000 00000000 00000001 ee8c3fc8 c01092e8 c01092ec 60000153 ffffffff
[??? 4.839156] [<c010d570>] (__irq_svc) from [<c01092ec>] (arch_cpu_idle+0x20/0x3c)
[??? 4.846527] [<c01092ec>] (arch_cpu_idle) from [<c015dbe4>] (do_idle+0x144/0x214)
[??? 4.853900] [<c015dbe4>] (do_idle) from [<c015dff0>] (cpu_startup_entry+0x18/0x1c)
[??? 4.861448] [<c015dff0>] (cpu_startup_entry) from [<4010190c>] (0x4010190c)
[??? 4.868436] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[??? 4.868436]

What can I do to help fixing it?

Quoted 63 lines
---
  arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
  arch/arm/kernel/entry-header.S   |  6 ++++++
  2 files changed, 24 insertions(+)
arch/arm/include/asm/assembler.h +18 -0
index ad301f107dd2..bc8d4bbd82e2 100644
@@ -518,4 +518,22 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
  #endif
  	.endm
 
+	.macro	bug, msg, line
+#ifdef CONFIG_THUMB2_KERNEL
+1:	.inst	0xde02
+#else
+1:	.inst	0xe7f001f2
+#endif
+#ifdef CONFIG_DEBUG_BUGVERBOSE
+	.pushsection .rodata.str, "aMS", %progbits, 1
+2:	.asciz	"\msg"
+	.popsection
+	.pushsection __bug_table, "aw"
+	.align	2
+	.word	1b, 2b
+	.hword	\line
+	.popsection
+#endif
+	.endm
+
  #endif /* __ASM_ASSEMBLER_H__ */
arch/arm/kernel/entry-header.S +5 -0
index d523cd8439a3..7f4d80c2db6b 100644
@@ -300,6 +300,8 @@
  	mov	r2, sp
  	ldr	r1, [r2, #\offset + S_PSR]	@ get calling cpsr
  	ldr	lr, [r2, #\offset + S_PC]!	@ get pc
+	tst	r1, #0xcf
+	bne	1f
  	msr	spsr_cxsf, r1			@ save in spsr_svc
  #if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K)
  	@ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -314,6 +316,7 @@
  						@ after ldm {}^
  	add	sp, sp, #\offset + PT_REGS_SIZE
  	movs	pc, lr				@ return & move spsr_svc into cpsr
+1:	bug	"Returning to usermode but unexpected PSR bits set?", \@
  #elif defined(CONFIG_CPU_V7M)
  	@ V7M restore.
  	@ Note that we don't need to do clrex here as clearing the local
@@ -329,6 +332,8 @@
  	ldr	r1, [sp, #\offset + S_PSR]	@ get calling cpsr
  	ldr	lr, [sp, #\offset + S_PC]	@ get pc
  	add	sp, sp, #\offset + S_SP
+	tst	r1, #0xcf
+	bne	1f
  	msr	spsr_cxsf, r1			@ save in spsr_svc
 
  	@ We must avoid clrex due to Cortex-A15 erratum #830321

arch/arm/kernel/entry-header.S line 352

Quoted 7 lines
arch/arm/kernel/entry-header.S +1 -0
@@ -341,6 +346,7 @@
  	.endif
  	add	sp, sp, #PT_REGS_SIZE - S_SP
  	movs	pc, lr				@ return & move spsr_svc into cpsr
+1:	bug	"Returning to usermode but unexpected PSR bits set?", \@
  #endif	/* !CONFIG_THUMB2_KERNEL */
  	.endm
 

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
author maintainer
#9 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 8 lines · Marek Szyprowski

Dear All,

Quoted 5 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

Quoted 5 lines
Quoted 3 lines

This ensures that we catch the problem case before we get to user code.

This patch brakes kernel booting on Exynos 4412-based Odroid U3 board:

"brakes" are what you have on a car. "breaks" is the verb, which is what you mean here.

It's known as of this morning, and I have a patch going through the build testers at the moment.

Just replace the 0xcf in these with PSR_I_BIT | 0x0f.

Also wonder why your platform is running with SCR.FW clear...

Signature · 3 lines
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
#10 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Hi Russel,

Quoted 9 lines · Russell King - ARM Linux
Quoted 8 lines · Marek Szyprowski

Dear All,

Quoted 5 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

Quoted 6 lines
Quoted 4 lines
Quoted 3 lines

This ensures that we catch the problem case before we get to user code.

This patch brakes kernel booting on Exynos 4412-based Odroid U3 board:

"brakes" are what you have on a car. "breaks" is the verb, which is what you mean here.

I'm sorry, I wrote that mail in a hurry before leaving the office.

Quoted 6 lines

It's known as of this morning, and I have a patch going through the build testers at the moment.

Just replace the 0xcf in these with PSR_I_BIT | 0x0f.

Also wonder why your platform is running with SCR.FW clear...

Yes, this fixes the issue. Thanks.

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
#11 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

In reply to Russell King

CC GregKH, since this patch would better to get to stable kernel. This patch also expose a qemu issue which fixed by upstream commit:
3aaf33bebda8d4ffcc0f ARM: avoid faulting on qemu

Both of patches are tested in linaro's kernelci and LKFT on stable kernel 4.4/4.9/4.14, no regressions. The lts 3.18 isn't applicable for this patches.

Regards
Alex

Quoted 79 lines · Russell King

Detect if we are returning to usermode via the normal kernel exit paths but the saved PSR value indicates that we are in kernel mode. This could occur due to corrupted stack state, which has been observed with "ftracetest".

This ensures that we catch the problem case before we get to user code.

---
 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/kernel/entry-header.S   |  6 ++++++
 2 files changed, 24 insertions(+)
arch/arm/include/asm/assembler.h +18 -0
index ad301f107dd2..bc8d4bbd82e2 100644
@@ -518,4 +518,22 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 #endif
 	.endm
 
+	.macro	bug, msg, line
+#ifdef CONFIG_THUMB2_KERNEL
+1:	.inst	0xde02
+#else
+1:	.inst	0xe7f001f2
+#endif
+#ifdef CONFIG_DEBUG_BUGVERBOSE
+	.pushsection .rodata.str, "aMS", %progbits, 1
+2:	.asciz	"\msg"
+	.popsection
+	.pushsection __bug_table, "aw"
+	.align	2
+	.word	1b, 2b
+	.hword	\line
+	.popsection
+#endif
+	.endm
+
 #endif /* __ASM_ASSEMBLER_H__ */
arch/arm/kernel/entry-header.S +6 -0
index d523cd8439a3..7f4d80c2db6b 100644
@@ -300,6 +300,8 @@
 	mov	r2, sp
 	ldr	r1, [r2, #\offset + S_PSR]	@ get calling cpsr
 	ldr	lr, [r2, #\offset + S_PC]!	@ get pc
+	tst	r1, #0xcf
+	bne	1f
 	msr	spsr_cxsf, r1			@ save in spsr_svc
 #if defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_32v6K)
 	@ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -314,6 +316,7 @@
 						@ after ldm {}^
 	add	sp, sp, #\offset + PT_REGS_SIZE
 	movs	pc, lr				@ return & move spsr_svc into cpsr
+1:	bug	"Returning to usermode but unexpected PSR bits set?", \@
 #elif defined(CONFIG_CPU_V7M)
 	@ V7M restore.
 	@ Note that we don't need to do clrex here as clearing the local
@@ -329,6 +332,8 @@
 	ldr	r1, [sp, #\offset + S_PSR]	@ get calling cpsr
 	ldr	lr, [sp, #\offset + S_PC]	@ get pc
 	add	sp, sp, #\offset + S_SP
+	tst	r1, #0xcf
+	bne	1f
 	msr	spsr_cxsf, r1			@ save in spsr_svc
 
 	@ We must avoid clrex due to Cortex-A15 erratum #830321
@@ -341,6 +346,7 @@
 	.endif
 	add	sp, sp, #PT_REGS_SIZE - S_SP
 	movs	pc, lr				@ return & move spsr_svc into cpsr
+1:	bug	"Returning to usermode but unexpected PSR bits set?", \@
 #endif	/* !CONFIG_THUMB2_KERNEL */
 	.endm
 
maintainer
#12 lore raw

PATCH 1/2 ARM: BUG if jumping to usermode address in kernel mode

Quoted 7 lines · Alex Shi

CC GregKH, since this patch would better to get to stable kernel. This patch also expose a qemu issue which fixed by upstream commit:
3aaf33bebda8d4ffcc0f ARM: avoid faulting on qemu

Both of patches are tested in linaro's kernelci and LKFT on stable kernel 4.4/4.9/4.14, no regressions. The lts 3.18 isn't applicable for this patches.

<formletter>

This is not the correct way to submit patches for inclusion in the stable kernel tree. Please read:
https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
for how to do this properly.

</formletter>

#13 lore raw

PATCH 2/2 ARM: probes: avoid adding kprobes to sensitive kernel-entry/exit code

In reply to Russell King - ARM Linux

Avoid adding kprobes to any of the kernel entry/exit or startup assembly code, or code in the identity-mapped region. This code does not conform to the standard C conventions, which means that the expectations of the kprobes code is not forfilled.

Placing kprobes at some of these locations results in the kernel trying to return to userspace addresses while retaining the CPU in kernel mode.

---
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 10 files changed, 44 insertions(+), 42 deletions(-)
arch/arm/include/asm/exception.h +1 -2
index a7273ad9587a..58e039a851af 100644
@@ -10,11 +10,10 @@
 
 #include <linux/interrupt.h>
 
-#define __exception	__attribute__((section(".exception.text")))
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 #define __exception_irq_entry	__irq_entry
 #else
-#define __exception_irq_entry	__exception
+#define __exception_irq_entry
 #endif
 
 #endif /* __ASM_ARM_EXCEPTION_H */
arch/arm/include/asm/sections.h +21 -0
index 63dfe1f10335..4ceb4f757d4d 100644
@@ -6,4 +6,25 @@
 
 extern char _exiprom[];
 
+extern char __idmap_text_start[];
+extern char __idmap_text_end[];
+extern char __entry_text_start[];
+extern char __entry_text_end[];
+extern char __hyp_idmap_text_start[];
+extern char __hyp_idmap_text_end[];
+
+static inline bool in_entry_text(unsigned long addr)
+{
+	return memory_contains(__entry_text_start, __entry_text_end,
+			       (void *)addr, 1);
+}
+
+static inline bool in_idmap_text(unsigned long addr)
+{
+	void *a = (void *)addr;
+	return memory_contains(__idmap_text_start, __idmap_text_end, a, 1) ||
+	       memory_contains(__hyp_idmap_text_start, __hyp_idmap_text_end,
+			       a, 1);
+}
+
 #endif	/* _ASM_ARM_SECTIONS_H */
arch/arm/include/asm/traps.h +0 -12
index f9a6c5fc3fd1..a00288d75ee6 100644
@@ -28,18 +28,6 @@ static inline int __in_irqentry_text(unsigned long ptr)
 	       ptr < (unsigned long)&__irqentry_text_end;
 }
 
-static inline int in_exception_text(unsigned long ptr)
-{
-	extern char __exception_text_start[];
-	extern char __exception_text_end[];
-	int in;
-
-	in = ptr >= (unsigned long)&__exception_text_start &&
-	     ptr < (unsigned long)&__exception_text_end;
-
-	return in ? : __in_irqentry_text(ptr);
-}
-
 extern void __init early_trap_init(void *);
 extern void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long frame);
 extern void ptrace_break(struct task_struct *tsk, struct pt_regs *regs);
arch/arm/kernel/entry-armv.S +1 -5
index 10ad44f3886a..b8ab97dfdd17 100644
@@ -82,11 +82,7 @@
 #endif
 	.endm
 
-#ifdef CONFIG_KPROBES
-	.section	.kprobes.text,"ax",%progbits
-#else
-	.text
-#endif
+	.section	.entry.text,"ax",%progbits
 
 /*
  * Invalid mode handlers
arch/arm/kernel/entry-common.S +1 -0
index e655dcd0a933..3c4f88701f22 100644
@@ -37,6 +37,7 @@ saved_pc	.req	lr
 #define TRACE(x...)
 #endif
 
+	.section .entry.text,"ax",%progbits
 	.align	5
 #if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING))
 /*
arch/arm/kernel/stacktrace.c +2 -12
index 65228bf4c6df..a56e7c856ab5 100644
@@ -3,6 +3,7 @@
 #include <linux/sched/debug.h>
 #include <linux/stacktrace.h>
 
+#include <asm/sections.h>
 #include <asm/stacktrace.h>
 #include <asm/traps.h>
 
@@ -63,7 +64,6 @@ EXPORT_SYMBOL(walk_stackframe);
 #ifdef CONFIG_STACKTRACE
 struct stack_trace_data {
 	struct stack_trace *trace;
-	unsigned long last_pc;
 	unsigned int no_sched_functions;
 	unsigned int skip;
 };
@@ -87,16 +87,7 @@ static int save_trace(struct stackframe *frame, void *d)
 	if (trace->nr_entries >= trace->max_entries)
 		return 1;
 
-	/*
-	 * in_exception_text() is designed to test if the PC is one of
-	 * the functions which has an exception stack above it, but
-	 * unfortunately what is in frame->pc is the return LR value,
-	 * not the saved PC value.  So, we need to track the previous
-	 * frame PC value when doing this.
-	 */
-	addr = data->last_pc;
-	data->last_pc = frame->pc;
-	if (!in_exception_text(addr))
+	if (!in_entry_text(frame->pc))
 		return 0;
 
 	regs = (struct pt_regs *)frame->sp;
@@ -114,7 +105,6 @@ static noinline void __save_stack_trace(struct task_struct *tsk,
 	struct stackframe frame;
 
 	data.trace = trace;
-	data.last_pc = ULONG_MAX;
 	data.skip = trace->skip;
 	data.no_sched_functions = nosched;
 
arch/arm/kernel/traps.c +2 -2
index 5de2bc46153f..95978073db53 100644
@@ -73,7 +73,7 @@ void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long
 	printk("Function entered at [<%08lx>] from [<%08lx>]\n", where, from);
 #endif
 
-	if (in_exception_text(where))
+	if (in_entry_text(from))
 		dump_mem("", "Exception stack", frame + 4, frame + 4 + sizeof(struct pt_regs));
 }
 
@@ -434,7 +434,7 @@ static int call_undef_hook(struct pt_regs *regs, unsigned int instr)
 	return fn ? fn(regs, instr) : 1;
 }
 
-asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
+asmlinkage void do_undefinstr(struct pt_regs *regs)
 {
 	unsigned int instr;
 	siginfo_t info;
arch/arm/kernel/vmlinux.lds.S +3 -3
index ee53f6518872..84a1ae3ce46e 100644
@@ -105,9 +105,9 @@ SECTIONS
 	.text : {			/* Real text segment		*/
 		_stext = .;		/* Text and read-only data	*/
 			IDMAP_TEXT
-			__exception_text_start = .;
-			*(.exception.text)
-			__exception_text_end = .;
+			__entry_text_start = .;
+			*(.entry.text)
+			__entry_text_end = .;
 			IRQENTRY_TEXT
 			SOFTIRQENTRY_TEXT
 			TEXT_TEXT
arch/arm/mm/fault.c +2 -3
index 42f585379e19..b75eada23d0a 100644
@@ -21,7 +21,6 @@
 #include <linux/highmem.h>
 #include <linux/perf_event.h>
 
-#include <asm/exception.h>
 #include <asm/pgtable.h>
 #include <asm/system_misc.h>
 #include <asm/system_info.h>
@@ -545,7 +544,7 @@ hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *)
 /*
  * Dispatch a data abort to the relevant handler.
  */
-asmlinkage void __exception
+asmlinkage void
 do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 {
 	const struct fsr_info *inf = fsr_info + fsr_fs(fsr);
@@ -578,7 +577,7 @@ hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *
 	ifsr_info[nr].name = name;
 }
 
-asmlinkage void __exception
+asmlinkage void
 do_PrefetchAbort(unsigned long addr, unsigned int ifsr, struct pt_regs *regs)
 {
 	const struct fsr_info *inf = ifsr_info + fsr_fs(ifsr);
arch/arm/probes/kprobes/core.c +11 -3
index 52d1cd14fda4..e90cc8a08186 100644
@@ -32,6 +32,7 @@
 #include <linux/percpu.h>
 #include <linux/bug.h>
 #include <asm/patch.h>
+#include <asm/sections.h>
 
 #include "../decode-arm.h"
 #include "../decode-thumb.h"
@@ -64,9 +65,6 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p)
 	int is;
 	const struct decode_checker **checkers;
 
-	if (in_exception_text(addr))
-		return -EINVAL;
-
 #ifdef CONFIG_THUMB2_KERNEL
 	thumb = true;
 	addr &= ~1; /* Bit 0 would normally be set to indicate Thumb code */
@@ -680,3 +678,13 @@ int __init arch_init_kprobes()
 #endif
 	return 0;
 }
+
+bool arch_within_kprobe_blacklist(unsigned long addr)
+{
+	void *a = (void *)addr;
+
+	return __in_irqentry_text(addr) ||
+	       in_entry_text(addr) ||
+	       in_idmap_text(addr) ||
+	       memory_contains(__kprobes_text_start, __kprobes_text_end, a, 1);
+}
Signature · 1 line
2.7.4
#14 lore raw

PATCH 2/2 ARM: probes: avoid adding kprobes to sensitive kernel-entry/exit code

Quoted 276 lines · Russell King

Avoid adding kprobes to any of the kernel entry/exit or startup assembly code, or code in the identity-mapped region. This code does not conform to the standard C conventions, which means that the expectations of the kprobes code is not forfilled.

Placing kprobes at some of these locations results in the kernel trying to return to userspace addresses while retaining the CPU in kernel mode.

---
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 10 files changed, 44 insertions(+), 42 deletions(-)
arch/arm/include/asm/exception.h +1 -2
index a7273ad9587a..58e039a851af 100644
@@ -10,11 +10,10 @@
 
 #include <linux/interrupt.h>
 
-#define __exception    __attribute__((section(".exception.text")))
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 #define __exception_irq_entry  __irq_entry
 #else
-#define __exception_irq_entry  __exception
+#define __exception_irq_entry
 #endif
 
 #endif /* __ASM_ARM_EXCEPTION_H */
arch/arm/include/asm/sections.h +21 -0
index 63dfe1f10335..4ceb4f757d4d 100644
@@ -6,4 +6,25 @@
 
 extern char _exiprom[];
 
+extern char __idmap_text_start[];
+extern char __idmap_text_end[];
+extern char __entry_text_start[];
+extern char __entry_text_end[];
+extern char __hyp_idmap_text_start[];
+extern char __hyp_idmap_text_end[];
+
+static inline bool in_entry_text(unsigned long addr)
+{
+       return memory_contains(__entry_text_start, __entry_text_end,
+                              (void *)addr, 1);
+}
+
+static inline bool in_idmap_text(unsigned long addr)
+{
+       void *a = (void *)addr;
+       return memory_contains(__idmap_text_start, __idmap_text_end, a, 1) ||
+              memory_contains(__hyp_idmap_text_start, __hyp_idmap_text_end,
+                              a, 1);
+}
+
 #endif /* _ASM_ARM_SECTIONS_H */
arch/arm/include/asm/traps.h +0 -12
index f9a6c5fc3fd1..a00288d75ee6 100644
@@ -28,18 +28,6 @@ static inline int __in_irqentry_text(unsigned long ptr)
               ptr < (unsigned long)&__irqentry_text_end;
 }
 
-static inline int in_exception_text(unsigned long ptr)
-{
-       extern char __exception_text_start[];
-       extern char __exception_text_end[];
-       int in;
-
-       in = ptr >= (unsigned long)&__exception_text_start &&
-            ptr < (unsigned long)&__exception_text_end;
-
-       return in ? : __in_irqentry_text(ptr);
-}
-
 extern void __init early_trap_init(void *);
 extern void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long frame);
 extern void ptrace_break(struct task_struct *tsk, struct pt_regs *regs);
arch/arm/kernel/entry-armv.S +1 -5
index 10ad44f3886a..b8ab97dfdd17 100644
@@ -82,11 +82,7 @@
 #endif
        .endm
 
-#ifdef CONFIG_KPROBES
-       .section        .kprobes.text,"ax",%progbits
-#else
-       .text
-#endif
+       .section        .entry.text,"ax",%progbits
 
 /*
  * Invalid mode handlers
arch/arm/kernel/entry-common.S +1 -0
index e655dcd0a933..3c4f88701f22 100644
@@ -37,6 +37,7 @@ saved_pc      .req    lr
 #define TRACE(x...)
 #endif
 
+       .section .entry.text,"ax",%progbits
        .align  5
 #if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING))
 /*
arch/arm/kernel/stacktrace.c +2 -12
index 65228bf4c6df..a56e7c856ab5 100644
@@ -3,6 +3,7 @@
 #include <linux/sched/debug.h>
 #include <linux/stacktrace.h>
 
+#include <asm/sections.h>
 #include <asm/stacktrace.h>
 #include <asm/traps.h>
 
@@ -63,7 +64,6 @@ EXPORT_SYMBOL(walk_stackframe);
 #ifdef CONFIG_STACKTRACE
 struct stack_trace_data {
        struct stack_trace *trace;
-       unsigned long last_pc;
        unsigned int no_sched_functions;
        unsigned int skip;
 };
@@ -87,16 +87,7 @@ static int save_trace(struct stackframe *frame, void *d)
        if (trace->nr_entries >= trace->max_entries)
                return 1;
 
-       /*
-        * in_exception_text() is designed to test if the PC is one of
-        * the functions which has an exception stack above it, but
-        * unfortunately what is in frame->pc is the return LR value,
-        * not the saved PC value.  So, we need to track the previous
-        * frame PC value when doing this.
-        */
-       addr = data->last_pc;
-       data->last_pc = frame->pc;
-       if (!in_exception_text(addr))
+       if (!in_entry_text(frame->pc))
                return 0;
 
        regs = (struct pt_regs *)frame->sp;
@@ -114,7 +105,6 @@ static noinline void __save_stack_trace(struct task_struct *tsk,
        struct stackframe frame;
 
        data.trace = trace;
-       data.last_pc = ULONG_MAX;
        data.skip = trace->skip;
        data.no_sched_functions = nosched;
 
arch/arm/kernel/traps.c +2 -2
index 5de2bc46153f..95978073db53 100644
@@ -73,7 +73,7 @@ void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long
        printk("Function entered at [<%08lx>] from [<%08lx>]\n", where, from);
 #endif
 
-       if (in_exception_text(where))
+       if (in_entry_text(from))
                dump_mem("", "Exception stack", frame + 4, frame + 4 + sizeof(struct pt_regs));
 }
 
@@ -434,7 +434,7 @@ static int call_undef_hook(struct pt_regs *regs, unsigned int instr)
        return fn ? fn(regs, instr) : 1;
 }
 
-asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
+asmlinkage void do_undefinstr(struct pt_regs *regs)
 {
        unsigned int instr;
        siginfo_t info;
arch/arm/kernel/vmlinux.lds.S +3 -3
index ee53f6518872..84a1ae3ce46e 100644
@@ -105,9 +105,9 @@ SECTIONS
        .text : {                       /* Real text segment            */
                _stext = .;             /* Text and read-only data      */
                        IDMAP_TEXT
-                       __exception_text_start = .;
-                       *(.exception.text)
-                       __exception_text_end = .;
+                       __entry_text_start = .;
+                       *(.entry.text)
+                       __entry_text_end = .;
                        IRQENTRY_TEXT
                        SOFTIRQENTRY_TEXT
                        TEXT_TEXT
arch/arm/mm/fault.c +2 -3
index 42f585379e19..b75eada23d0a 100644
@@ -21,7 +21,6 @@
 #include <linux/highmem.h>
 #include <linux/perf_event.h>
 
-#include <asm/exception.h>
 #include <asm/pgtable.h>
 #include <asm/system_misc.h>
 #include <asm/system_info.h>
@@ -545,7 +544,7 @@ hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *)
 /*
  * Dispatch a data abort to the relevant handler.
  */
-asmlinkage void __exception
+asmlinkage void
 do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 {
        const struct fsr_info *inf = fsr_info + fsr_fs(fsr);
@@ -578,7 +577,7 @@ hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *
        ifsr_info[nr].name = name;
 }
 
-asmlinkage void __exception
+asmlinkage void
 do_PrefetchAbort(unsigned long addr, unsigned int ifsr, struct pt_regs *regs)
 {
        const struct fsr_info *inf = ifsr_info + fsr_fs(ifsr);
arch/arm/probes/kprobes/core.c +11 -4
index 52d1cd14fda4..e90cc8a08186 100644
@@ -32,6 +32,7 @@
 #include <linux/percpu.h>
 #include <linux/bug.h>
 #include <asm/patch.h>
+#include <asm/sections.h>
 
 #include "../decode-arm.h"
 #include "../decode-thumb.h"
@@ -64,9 +65,6 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p)
        int is;
        const struct decode_checker **checkers;
 
-       if (in_exception_text(addr))
-               return -EINVAL;
-
 #ifdef CONFIG_THUMB2_KERNEL
        thumb = true;
        addr &= ~1; /* Bit 0 would normally be set to indicate Thumb code */
@@ -680,3 +678,13 @@ int __init arch_init_kprobes()
 #endif
        return 0;
 }
+
+bool arch_within_kprobe_blacklist(unsigned long addr)
+{
+       void *a = (void *)addr;
+
+       return __in_irqentry_text(addr) ||
+              in_entry_text(addr) ||
+              in_idmap_text(addr) ||
+              memory_contains(__kprobes_text_start, __kprobes_text_end, a, 1);
+}
--

2.7.4

Quoted 4 lines

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Hi Russel,

Can you please tell us what is the status of this patch? It fixes the issue for us ([1]), but we are waiting for it to be merged.

Thanks!

[1] https://bugs.linaro.org/show_bug.cgi?id=3297

author maintainer
#15 lore raw

PATCH 2/2 ARM: probes: avoid adding kprobes to sensitive kernel-entry/exit code

Quoted 277 lines · Sam Protsenko
Quoted 276 lines · Russell King

Avoid adding kprobes to any of the kernel entry/exit or startup assembly code, or code in the identity-mapped region. This code does not conform to the standard C conventions, which means that the expectations of the kprobes code is not forfilled.

Placing kprobes at some of these locations results in the kernel trying to return to userspace addresses while retaining the CPU in kernel mode.

---
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 10 files changed, 44 insertions(+), 42 deletions(-)
arch/arm/include/asm/exception.h +1 -2
index a7273ad9587a..58e039a851af 100644
@@ -10,11 +10,10 @@
 
 #include <linux/interrupt.h>
 
-#define __exception    __attribute__((section(".exception.text")))
 #ifdef CONFIG_FUNCTION_GRAPH_TRACER
 #define __exception_irq_entry  __irq_entry
 #else
-#define __exception_irq_entry  __exception
+#define __exception_irq_entry
 #endif
 
 #endif /* __ASM_ARM_EXCEPTION_H */
arch/arm/include/asm/sections.h +21 -0
index 63dfe1f10335..4ceb4f757d4d 100644
@@ -6,4 +6,25 @@
 
 extern char _exiprom[];
 
+extern char __idmap_text_start[];
+extern char __idmap_text_end[];
+extern char __entry_text_start[];
+extern char __entry_text_end[];
+extern char __hyp_idmap_text_start[];
+extern char __hyp_idmap_text_end[];
+
+static inline bool in_entry_text(unsigned long addr)
+{
+       return memory_contains(__entry_text_start, __entry_text_end,
+                              (void *)addr, 1);
+}
+
+static inline bool in_idmap_text(unsigned long addr)
+{
+       void *a = (void *)addr;
+       return memory_contains(__idmap_text_start, __idmap_text_end, a, 1) ||
+              memory_contains(__hyp_idmap_text_start, __hyp_idmap_text_end,
+                              a, 1);
+}
+
 #endif /* _ASM_ARM_SECTIONS_H */
arch/arm/include/asm/traps.h +0 -12
index f9a6c5fc3fd1..a00288d75ee6 100644
@@ -28,18 +28,6 @@ static inline int __in_irqentry_text(unsigned long ptr)
               ptr < (unsigned long)&__irqentry_text_end;
 }
 
-static inline int in_exception_text(unsigned long ptr)
-{
-       extern char __exception_text_start[];
-       extern char __exception_text_end[];
-       int in;
-
-       in = ptr >= (unsigned long)&__exception_text_start &&
-            ptr < (unsigned long)&__exception_text_end;
-
-       return in ? : __in_irqentry_text(ptr);
-}
-
 extern void __init early_trap_init(void *);
 extern void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long frame);
 extern void ptrace_break(struct task_struct *tsk, struct pt_regs *regs);
arch/arm/kernel/entry-armv.S +1 -5
index 10ad44f3886a..b8ab97dfdd17 100644
@@ -82,11 +82,7 @@
 #endif
        .endm
 
-#ifdef CONFIG_KPROBES
-       .section        .kprobes.text,"ax",%progbits
-#else
-       .text
-#endif
+       .section        .entry.text,"ax",%progbits
 
 /*
  * Invalid mode handlers
arch/arm/kernel/entry-common.S +1 -0
index e655dcd0a933..3c4f88701f22 100644
@@ -37,6 +37,7 @@ saved_pc      .req    lr
 #define TRACE(x...)
 #endif
 
+       .section .entry.text,"ax",%progbits
        .align  5
 #if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING))
 /*
arch/arm/kernel/stacktrace.c +2 -12
index 65228bf4c6df..a56e7c856ab5 100644
@@ -3,6 +3,7 @@
 #include <linux/sched/debug.h>
 #include <linux/stacktrace.h>
 
+#include <asm/sections.h>
 #include <asm/stacktrace.h>
 #include <asm/traps.h>
 
@@ -63,7 +64,6 @@ EXPORT_SYMBOL(walk_stackframe);
 #ifdef CONFIG_STACKTRACE
 struct stack_trace_data {
        struct stack_trace *trace;
-       unsigned long last_pc;
        unsigned int no_sched_functions;
        unsigned int skip;
 };
@@ -87,16 +87,7 @@ static int save_trace(struct stackframe *frame, void *d)
        if (trace->nr_entries >= trace->max_entries)
                return 1;
 
-       /*
-        * in_exception_text() is designed to test if the PC is one of
-        * the functions which has an exception stack above it, but
-        * unfortunately what is in frame->pc is the return LR value,
-        * not the saved PC value.  So, we need to track the previous
-        * frame PC value when doing this.
-        */
-       addr = data->last_pc;
-       data->last_pc = frame->pc;
-       if (!in_exception_text(addr))
+       if (!in_entry_text(frame->pc))
                return 0;
 
        regs = (struct pt_regs *)frame->sp;
@@ -114,7 +105,6 @@ static noinline void __save_stack_trace(struct task_struct *tsk,
        struct stackframe frame;
 
        data.trace = trace;
-       data.last_pc = ULONG_MAX;
        data.skip = trace->skip;
        data.no_sched_functions = nosched;
 
arch/arm/kernel/traps.c +2 -2
index 5de2bc46153f..95978073db53 100644
@@ -73,7 +73,7 @@ void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long
        printk("Function entered at [<%08lx>] from [<%08lx>]\n", where, from);
 #endif
 
-       if (in_exception_text(where))
+       if (in_entry_text(from))
                dump_mem("", "Exception stack", frame + 4, frame + 4 + sizeof(struct pt_regs));
 }
 
@@ -434,7 +434,7 @@ static int call_undef_hook(struct pt_regs *regs, unsigned int instr)
        return fn ? fn(regs, instr) : 1;
 }
 
-asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
+asmlinkage void do_undefinstr(struct pt_regs *regs)
 {
        unsigned int instr;
        siginfo_t info;
arch/arm/kernel/vmlinux.lds.S +3 -3
index ee53f6518872..84a1ae3ce46e 100644
@@ -105,9 +105,9 @@ SECTIONS
        .text : {                       /* Real text segment            */
                _stext = .;             /* Text and read-only data      */
                        IDMAP_TEXT
-                       __exception_text_start = .;
-                       *(.exception.text)
-                       __exception_text_end = .;
+                       __entry_text_start = .;
+                       *(.entry.text)
+                       __entry_text_end = .;
                        IRQENTRY_TEXT
                        SOFTIRQENTRY_TEXT
                        TEXT_TEXT
arch/arm/mm/fault.c +2 -3
index 42f585379e19..b75eada23d0a 100644
@@ -21,7 +21,6 @@
 #include <linux/highmem.h>
 #include <linux/perf_event.h>
 
-#include <asm/exception.h>
 #include <asm/pgtable.h>
 #include <asm/system_misc.h>
 #include <asm/system_info.h>
@@ -545,7 +544,7 @@ hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *)
 /*
  * Dispatch a data abort to the relevant handler.
  */
-asmlinkage void __exception
+asmlinkage void
 do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 {
        const struct fsr_info *inf = fsr_info + fsr_fs(fsr);
@@ -578,7 +577,7 @@ hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *
        ifsr_info[nr].name = name;
 }
 
-asmlinkage void __exception
+asmlinkage void
 do_PrefetchAbort(unsigned long addr, unsigned int ifsr, struct pt_regs *regs)
 {
        const struct fsr_info *inf = ifsr_info + fsr_fs(ifsr);
arch/arm/probes/kprobes/core.c +11 -4
index 52d1cd14fda4..e90cc8a08186 100644
@@ -32,6 +32,7 @@
 #include <linux/percpu.h>
 #include <linux/bug.h>
 #include <asm/patch.h>
+#include <asm/sections.h>
 
 #include "../decode-arm.h"
 #include "../decode-thumb.h"
@@ -64,9 +65,6 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p)
        int is;
        const struct decode_checker **checkers;
 
-       if (in_exception_text(addr))
-               return -EINVAL;
-
 #ifdef CONFIG_THUMB2_KERNEL
        thumb = true;
        addr &= ~1; /* Bit 0 would normally be set to indicate Thumb code */
@@ -680,3 +678,13 @@ int __init arch_init_kprobes()
 #endif
        return 0;
 }
+
+bool arch_within_kprobe_blacklist(unsigned long addr)
+{
+       void *a = (void *)addr;
+
+       return __in_irqentry_text(addr) ||
+              in_entry_text(addr) ||
+              in_idmap_text(addr) ||
+              memory_contains(__kprobes_text_start, __kprobes_text_end, a, 1);
+}
--

2.7.4

Quoted 6 lines
Quoted 4 lines

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Hi Russel,

Hi Sa,

Quoted 8 lines

Can you please tell us what is the status of this patch? It fixes the issue for us ([1]), but we are waiting for it to be merged.

Thanks!

[1] https://bugs.linaro.org/show_bug.cgi?id=3297

It's queued for the next merge window because it's not a regression, but rather a large patch to fix a long standing bug. Given that the bug has been around for a long time (probably since kprobes was originally added), I see no reason to push it for the -rc kernels, especially given that it has taken about a month to get any kind of feedback on this particular patch.

Signature · 3 lines
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up
#16 lore raw

cover Fix ftracetest issues

In reply to Russell King - ARM Linux

CC Naresh,

Could you like to test the patch for our bug 3297?

BTW,
The 2nd has 2 build error on stm32_defconfig:

arch/arm/kernel/traps.o: In function `dump_backtrace_entry':
/mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_start' /mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_end'

Quoted 37 lines · Russell King - ARM Linux

ftracetest provokes the kernel to try and return to userspace addresses in kernel mode. These two patches prevent that.

The first patch, which I intend merging with Linus tonight, ensures that we catch the condition before we hit userspace, meaning that there is no possibility of executing user code while in kernel mode.

The second patch fixes the ftracetest issue itself by ensuring that it is not possible to set a kprobe on any of the "special" assembler code. Such code includes:
- the kernel primary/secondary CPU startup code
- exception entry code
- idmap code

This is because the conditions under which this code is executed does not meet the kprobes requirements, which is basically that the "function" is C-like - it does something and then returns to the parent, and has a stack. This is just not universally true of the above code.

This patch is larger than one may desire as we re-organise the sections that some code ends up in, the way the unwinder works, and how we print stack frames.

 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/entry-header.S   |  6 ++++++
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 12 files changed, 68 insertions(+), 42 deletions(-)
#17 lore raw

cover Fix ftracetest issues

Hi Alex Shi,

Quoted 4 lines · Alex Shi

CC Naresh,

Could you like to test the patch for our bug 3297?

Quoted 9 lines

BTW,
The 2nd has 2 build error on stm32_defconfig:

arch/arm/kernel/traps.o: In function `dump_backtrace_entry':
/mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_start' /mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_end'

Case 1: Build PASS on linux-4.14.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.14.y
Commit id: f9f0b03dedc1 (HEAD -> linux-4.14.y, tag: v4.14.2, origin/linux-4.14.y) Linux 4.14.2

Patches 1 and 2 applies smooth and build PASS with defconfig. Ftrace test did not hang on beagleboard x15.

Case 2: Build failed on linux-4.9.y and linux-4.4.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.9.y
Commit id: 133e6ccf46f1 (HEAD -> linux-4.9.y, tag: v4.9.65, origin/linux-4.9.y) Linux 4.9.65

Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.4.y
Commit id: 29ffb9c1fb4a (HEAD, tag: v4.4.102, origin/linux-4.4.y) Linux 4.4.102

Patch-1 : Patch applied smooth.
Patch-2: Patch failed to apply. Manually changed this file "arch/arm/include/asm/traps.h"

Build failed on linux-4.9.y and linux-4.4.y with defconfig.

Build Error
---------------
arch/arm/kernel/traps.c: In function ?dump_backtrace_entry?:
arch/arm/kernel/traps.c:73:2: error: implicit declaration of function ?in_entry_text? [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[1]: *** [arch/arm/kernel/traps.o] Error 1
make: *** [arch/arm/kernel] Error 2

- Naresh

Quoted 38 lines · Russell King - ARM Linux
Quoted 37 lines

ftracetest provokes the kernel to try and return to userspace addresses in kernel mode. These two patches prevent that.

The first patch, which I intend merging with Linus tonight, ensures that we catch the condition before we hit userspace, meaning that there is no possibility of executing user code while in kernel mode.

The second patch fixes the ftracetest issue itself by ensuring that it is not possible to set a kprobe on any of the "special" assembler code. Such code includes:
- the kernel primary/secondary CPU startup code
- exception entry code
- idmap code

This is because the conditions under which this code is executed does not meet the kprobes requirements, which is basically that the "function" is C-like - it does something and then returns to the parent, and has a stack. This is just not universally true of the above code.

This patch is larger than one may desire as we re-organise the sections that some code ends up in, the way the unwinder works, and how we print stack frames.

 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/entry-header.S   |  6 ++++++
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 12 files changed, 68 insertions(+), 42 deletions(-)
#18 lore raw

cover Fix ftracetest issues

Quoted 11 lines

Case 1: Build PASS on linux-4.14.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.14.y
Commit id: f9f0b03dedc1 (HEAD -> linux-4.14.y, tag: v4.14.2, origin/linux-4.14.y) Linux 4.14.2

Patches 1 and 2 applies smooth and build PASS with defconfig. Ftrace test did not hang on beagleboard x15.

Thanks Naresh, could you like to give a 'tested-by:' for theses patches?

Quoted 68 lines

Case 2: Build failed on linux-4.9.y and linux-4.4.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.9.y
Commit id: 133e6ccf46f1 (HEAD -> linux-4.9.y, tag: v4.9.65, origin/linux-4.9.y) Linux 4.9.65

Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.4.y
Commit id: 29ffb9c1fb4a (HEAD, tag: v4.4.102, origin/linux-4.4.y) Linux 4.4.102

Patch-1 : Patch applied smooth.
Patch-2: Patch failed to apply. Manually changed this file "arch/arm/include/asm/traps.h"

Build failed on linux-4.9.y and linux-4.4.y with defconfig.

Build Error
---------------
arch/arm/kernel/traps.c: In function ?dump_backtrace_entry?:
arch/arm/kernel/traps.c:73:2: error: implicit declaration of function ?in_entry_text? [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[1]: *** [arch/arm/kernel/traps.o] Error 1
make: *** [arch/arm/kernel] Error 2

- Naresh

Quoted 38 lines · Russell King - ARM Linux
Quoted 37 lines

ftracetest provokes the kernel to try and return to userspace addresses in kernel mode. These two patches prevent that.

The first patch, which I intend merging with Linus tonight, ensures that we catch the condition before we hit userspace, meaning that there is no possibility of executing user code while in kernel mode.

The second patch fixes the ftracetest issue itself by ensuring that it is not possible to set a kprobe on any of the "special" assembler code. Such code includes:
- the kernel primary/secondary CPU startup code
- exception entry code
- idmap code

This is because the conditions under which this code is executed does not meet the kprobes requirements, which is basically that the "function" is C-like - it does something and then returns to the parent, and has a stack. This is just not universally true of the above code.

This patch is larger than one may desire as we re-organise the sections that some code ends up in, the way the unwinder works, and how we print stack frames.

 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/entry-header.S   |  6 ++++++
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 12 files changed, 68 insertions(+), 42 deletions(-)
#19 lore raw

cover Fix ftracetest issues

In reply to Naresh Kamboju
Quoted 24 lines · Naresh Kamboju

Hi Alex Shi,

Quoted 4 lines · Alex Shi

CC Naresh,

Could you like to test the patch for our bug 3297?

Quoted 9 lines

BTW,
The 2nd has 2 build error on stm32_defconfig:

arch/arm/kernel/traps.o: In function `dump_backtrace_entry':
/mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_start' /mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_end'

Case 1: Build PASS on linux-4.14.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.14.y
Commit id: f9f0b03dedc1 (HEAD -> linux-4.14.y, tag: v4.14.2, origin/linux-4.14.y) Linux 4.14.2

Quoted 2 lines

Patches 1 and 2 applies smooth and build PASS with defconfig. Ftrace test did not hang on beagleboard x15.

- Naresh

Quoted 69 lines

Case 2: Build failed on linux-4.9.y and linux-4.4.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.9.y
Commit id: 133e6ccf46f1 (HEAD -> linux-4.9.y, tag: v4.9.65, origin/linux-4.9.y) Linux 4.9.65

Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.4.y
Commit id: 29ffb9c1fb4a (HEAD, tag: v4.4.102, origin/linux-4.4.y) Linux 4.4.102

Patch-1 : Patch applied smooth.
Patch-2: Patch failed to apply. Manually changed this file "arch/arm/include/asm/traps.h"

Build failed on linux-4.9.y and linux-4.4.y with defconfig.

Build Error
---------------
arch/arm/kernel/traps.c: In function ?dump_backtrace_entry?:
arch/arm/kernel/traps.c:73:2: error: implicit declaration of function ?in_entry_text? [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[1]: *** [arch/arm/kernel/traps.o] Error 1
make: *** [arch/arm/kernel] Error 2

- Naresh

Quoted 38 lines · Russell King - ARM Linux
Quoted 37 lines

ftracetest provokes the kernel to try and return to userspace addresses in kernel mode. These two patches prevent that.

The first patch, which I intend merging with Linus tonight, ensures that we catch the condition before we hit userspace, meaning that there is no possibility of executing user code while in kernel mode.

The second patch fixes the ftracetest issue itself by ensuring that it is not possible to set a kprobe on any of the "special" assembler code. Such code includes:
- the kernel primary/secondary CPU startup code
- exception entry code
- idmap code

This is because the conditions under which this code is executed does not meet the kprobes requirements, which is basically that the "function" is C-like - it does something and then returns to the parent, and has a stack. This is just not universally true of the above code.

This patch is larger than one may desire as we re-organise the sections that some code ends up in, the way the unwinder works, and how we print stack frames.

 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/entry-header.S   |  6 ++++++
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 12 files changed, 68 insertions(+), 42 deletions(-)
#20 lore raw

cover Fix ftracetest issues

Quoted 25 lines · Naresh Kamboju
Quoted 24 lines · Naresh Kamboju

Hi Alex Shi,

Quoted 4 lines · Alex Shi

CC Naresh,

Could you like to test the patch for our bug 3297?

Quoted 9 lines

BTW,
The 2nd has 2 build error on stm32_defconfig:

arch/arm/kernel/traps.o: In function `dump_backtrace_entry':
/mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_start' /mmkernels/kernel/arch/arm/kernel/traps.c:77: undefined reference to __entry_text_end'

Case 1: Build PASS on linux-4.14.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.14.y
Commit id: f9f0b03dedc1 (HEAD -> linux-4.14.y, tag: v4.14.2, origin/linux-4.14.y) Linux 4.14.2

Quoted 7 lines
Quoted 2 lines

Patches 1 and 2 applies smooth and build PASS with defconfig. Ftrace test did not hang on beagleboard x15.

- Naresh

This series fixed ftracetest on X15 board. After manually applying second patch (on linux-mainline kernel):

[PATCH 2/2] ARM: probes: avoid adding kprobes to sensitive
kernel-entry/exit code

all tests are passing. Some inputs:
- log without applying the patch: [1]
- log with patch applied: [2]
- corresponding bug report: [3]

Hope that patch will be merged soon.

Thanks.

[1] http://pastebin.ubuntu.com/26189372/
[2] http://pastebin.ubuntu.com/26189667/
[3] https://bugs.linaro.org/show_bug.cgi?id=3304

Quoted 74 lines
Quoted 29 lines

Case 2: Build failed on linux-4.9.y and linux-4.4.y
-----------
Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.9.y
Commit id: 133e6ccf46f1 (HEAD -> linux-4.9.y, tag: v4.9.65, origin/linux-4.9.y) Linux 4.9.65

Tree: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
Branch: linux-4.4.y
Commit id: 29ffb9c1fb4a (HEAD, tag: v4.4.102, origin/linux-4.4.y) Linux 4.4.102

Patch-1 : Patch applied smooth.
Patch-2: Patch failed to apply. Manually changed this file "arch/arm/include/asm/traps.h"

Build failed on linux-4.9.y and linux-4.4.y with defconfig.

Build Error
---------------
arch/arm/kernel/traps.c: In function ?dump_backtrace_entry?:
arch/arm/kernel/traps.c:73:2: error: implicit declaration of function ?in_entry_text? [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[1]: *** [arch/arm/kernel/traps.o] Error 1
make: *** [arch/arm/kernel] Error 2

Quoted 40 lines

- Naresh

Quoted 38 lines · Russell King - ARM Linux
Quoted 37 lines

ftracetest provokes the kernel to try and return to userspace addresses in kernel mode. These two patches prevent that.

The first patch, which I intend merging with Linus tonight, ensures that we catch the condition before we hit userspace, meaning that there is no possibility of executing user code while in kernel mode.

The second patch fixes the ftracetest issue itself by ensuring that it is not possible to set a kprobe on any of the "special" assembler code. Such code includes:
- the kernel primary/secondary CPU startup code
- exception entry code
- idmap code

This is because the conditions under which this code is executed does not meet the kprobes requirements, which is basically that the "function" is C-like - it does something and then returns to the parent, and has a stack. This is just not universally true of the above code.

This patch is larger than one may desire as we re-organise the sections that some code ends up in, the way the unwinder works, and how we print stack frames.

 arch/arm/include/asm/assembler.h | 18 ++++++++++++++++++
 arch/arm/include/asm/exception.h |  3 +--
 arch/arm/include/asm/sections.h  | 21 +++++++++++++++++++++
 arch/arm/include/asm/traps.h     | 12 ------------
 arch/arm/kernel/entry-armv.S     |  6 +-----
 arch/arm/kernel/entry-common.S   |  1 +
 arch/arm/kernel/entry-header.S   |  6 ++++++
 arch/arm/kernel/stacktrace.c     | 14 ++------------
 arch/arm/kernel/traps.c          |  4 ++--
 arch/arm/kernel/vmlinux.lds.S    |  6 +++---
 arch/arm/mm/fault.c              |  5 ++---
 arch/arm/probes/kprobes/core.c   | 14 +++++++++++---
 12 files changed, 68 insertions(+), 42 deletions(-)

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel