ARM: !MMU v7 Cortex-M preemption support
cover ARM: !MMU v7 Cortex-M preemption support
Hi,
[ cc'ing those who added long back !MMU support & Cortex-M support as
well, though some not working anymore on these ]
An attempt to add preemption support to the !MMU, ARM Cortex-M platform. With these changes, there is some effect, but not enough to make it usable.
Though as of now i don't have much clue on what the issue could be, it is being debugged. Meanwhile sending the series as RFC, to get feedback on what could be missing or whether there is anything inherent in !MMU and/or Cortex-M platforms that could prevent supporting preemption, etc.
Details on the problems, the way it was tested etc. are mentioned in 2/2. 1/2 is just a change made so as to have a clear diff for easy review of the preemption support changes.
Regards
afzal
afzal mohammed (2):
ARM: !MMU: v7-M: prepare preemption support
ARM: !MMU: v7-M: preemption support
arch/arm/kernel/entry-v7m.S | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-)
base-commit: fd6988496e79a6a4bdb514a4655d2920209eb85d
Signature · 1 line
2.24.1
RFC PATCH 1/2 ARM: !MMU: v7-M: prepare preemption support
Rearrange getting thread_info pointer & popping lr so as to have an easy to review diff for preempt support that is going to be added.
--- arch/arm/kernel/entry-v7m.S | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/arm/kernel/entry-v7m.S +3 -2
@@ -48,7 +48,7 @@ strerr: .asciz "\nUnhandled exception: IPSR = %08lx LR = %08lx\n" @ routine called with r0 = irq number, r1 = struct pt_regs * bl nvic_handle_irq - pop {lr} + get_thread_info tsk @ @ Check for any pending work if returning to user @
@@ -57,7 +57,6 @@ strerr: .asciz "\nUnhandled exception: IPSR = %08lx LR = %08lx\n" tst r0, V7M_SCB_ICSR_RETTOBASE beq 2f - get_thread_info tsk ldr r2, [tsk, #TI_FLAGS] tst r2, #_TIF_WORK_MASK beq 2f @ no work pending
@@ -65,6 +64,8 @@ strerr: .asciz "\nUnhandled exception: IPSR = %08lx LR = %08lx\n" str r0, [r1, V7M_SCB_ICSR] @ raise PendSV 2: + pop {lr} + @ registers r0-r3 and r12 are automatically restored on exception @ return. r4-r7 were not clobbered in v7m_exception_entry so for @ correctness they don't need to be restored. So only r8-r11 must be
Signature · 1 line
2.24.1
RFC PATCH 2/2 ARM: !MMU: v7-M: preemption support
In reply to afzal mohammedICSR RETTOBASE bit indicates whether there is any active exception other than the exception indicated by current value of IPSR. Existing code already makes use of it to find whether interrupt occurred during user(thread) mode. Add preemption support in the code path that executes upon finding an active exception other than the current interrupt.
Having an active exception during interrupt exception indicates that the interrupt occurred during SVC or PendSV with current Linux exception setup. On PendSV, interrupts are disabled at exception entry (applicable to all) & enabled back just before returning from exception. Hence it has been assumed that active exception during interrupt is indicating SVC interruption. This would have to be revisited.
With this change, there is some effect, but not enough to make it usable, this is being debugged.
A simple character driver that does infinite while loop during read was used to test it as follows (/dev/mymisc corresponds to the node for the character driver),
A. Before this change,
A.1
~ # cat /dev/mymisc
[ 28.099225] mymisc_open [ 28.101625] mymisc_read: enter
command prompt is not usable (expected), interrupts do happen during this time.
A.2
~ # cat /dev/mymisc &
[1] 39 cat /dev/mymisc
~ # [ 11.699880] mymisc_open
[ 11.702274] mymisc_read: enter
command prompt is not usable (expected as preemption not supported).
B. After this change,
B.1
~ # cat /dev/mymisc
[ 27.374821] mymisc_open [ 27.377349] mymisc_read: enter
though user will not get control back (as expected as it is fg process), entering on prompt causes new line, doesn't know what to make out of this behaviour, this doesn't happen in the A.1 case. Interrupts happen here as well.
B.2
~ # cat /dev/mymisc &
[1] 41 cat /dev/mymisc
~ # [ 44.836417] mymisc_open
[ 44.838814] mymisc_read: enter
though prompt is available under the control of user, upon typing anything on the prompt (typed character doesn't get echoed), it crashes as follows,
[ 44.838814] mymisc_read: enter [ 51.710314] [ 51.710314] Unhandled exception: IPSR = 00000006 LR = fffffffd [ 51.717576] CPU: 0 PID: 37 Comm: sh Not tainted 5.5.0-rc4-00004-g2328d01dbd85 #105 [ 51.725078] Hardware name: STM32 (Device Tree Support) [ 51.730206] PC is at 0x90195958 [ 51.733329] LR is at 0x901c4df3 [ 51.736471] pc : [<90195958>] lr : [<901c4df3>] psr: 21000000 [ 51.742713] sp : 901e5a58 ip : 00000000 fp : 901d89fc [ 51.747911] r10: 00000000 r9 : 00000000 r8 : 00000001 [ 51.753143] r7 : 000000a8 r6 : 901e5a58 r5 : 901e5b08 r4 : ffffffff [ 51.759643] r3 : 000000a8 r2 : ffffffff r1 : 00000001 r0 : 00000001 [ 51.766122] xPSR: 21000000 [ 51.768866] CPU: 0 PID: 37 Comm: sh Not tainted 5.5.0-rc4-00004-g2328d01dbd85 #105 [ 51.776369] Hardware name: STM32 (Device Tree Support) [ 51.781594] [<0800c0c9>] (unwind_backtrace) from [<0800b25b>] (show_stack+0xb/0xc) [ 51.789166] [<0800b25b>] (show_stack) from [<0800b9eb>] (__invalid_entry+0x4b/0x4c)
It is a Usage Fault happening while in thread(user) mode. PC & LR in the dump is strange in the sense that they do not point to text section.
The change here has been made based on how Cortex-A handles preemption. Since PAN is not applicable and since it seemed that saving & restoring thread_info::addr_limit does not make difference on the !MMU Cortex-M due to single address space being used, there was no additional save & restore of 'dacr' & 'addr_limit' as done in the case of interrupt during system call on Cortex-A. Also as DUT was UP Cortex-M4, it seemed that cache related would not affect in anyway w.r.t preemption, as currently cache is not enabled for M4 in Linux.
As mentioned above, it was assumed that active exception during interrupt meant SVC was interrupted, though it could have been PendSV as well. To ensure that issue is not due to interrupt during PendSV, the PendSV handling of pending work was lifted & sticked onto __irq_entry and setting PendSV removed as well. Still the behaviour is same.
--- arch/arm/kernel/entry-v7m.S | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
arch/arm/kernel/entry-v7m.S +16 -1
@@ -55,14 +55,29 @@ strerr: .asciz "\nUnhandled exception: IPSR = %08lx LR = %08lx\n" ldr r1, =BASEADDR_V7M_SCB ldr r0, [r1, V7M_SCB_ICSR] tst r0, V7M_SCB_ICSR_RETTOBASE - beq 2f + beq 101f ldr r2, [tsk, #TI_FLAGS] tst r2, #_TIF_WORK_MASK beq 2f @ no work pending mov r0, #V7M_SCB_ICSR_PENDSVSET str r0, [r1, V7M_SCB_ICSR] @ raise PendSV + b 2f + +101: +#ifdef CONFIG_PREEMPT + ldr r2, [tsk, #TI_PREEMPT] + teq r2, #0 @ preempt count != 0 ? + bne 2f + ldr r0, [tsk, #TI_FLAGS] + tst r0, #_TIF_NEED_RESCHED @ __TIF_NEED_RESCHED set ? + beq 2f +102: bl preempt_schedule_irq + ldr r0, [tsk, #TI_FLAGS] + tst r0, #_TIF_NEED_RESCHED @ __TIF_NEED_RESCHED set ? + bne 102b +#endif 2: pop {lr}
Signature · 1 line
2.24.1
Re: [RFC PATCH 2/2] ARM: !MMU: v7-M: preemption support
Hi,
Quoted 14 lines · afzal mohammed
A. Before this change,
A.1
~ # cat/dev/mymisc[ 28.099225] mymisc_open [ 28.101625] mymisc_read: entercommand prompt is not usable (expected), interrupts do happen during this time.
B. After this change,
B.1
~ # cat/dev/mymisc
Quoted 7 lines
[ 27.374821] mymisc_open [ 27.377349] mymisc_read: enterthough user will not get control back (as expected as it is fg process), entering on prompt causes new line, doesn't know what to make out of this behaviour, this doesn't happen in the A.1 case. Interrupts happen here as well.
Behaviour B.1 (which is how most of the system behaves) seems due to serial driver's threaded interrupt handler being able to preempt 'cat' process, this can't happen in the case of A.1 as preemption doesn't work.
Also it seems that preemption does happen by tracking __schedule() at runtime, but issue mentioned in B.2 (below) remains.
So it seems preemption is happening, but is very fragile.
Regards
afzal
Quoted 30 lines
B.2
~ # cat/dev/mymisc&
[1] 41 cat/dev/mymisc
~ # [ 44.836417]mymisc_open[ 44.838814] mymisc_read: enterthough prompt is available under the control of user, upon typing anything on the prompt (typed character doesn't get echoed), it crashes as follows,
[ 44.838814] mymisc_read: enter [ 51.710314] [ 51.710314] Unhandled exception: IPSR = 00000006 LR = fffffffd [ 51.717576] CPU: 0 PID: 37 Comm: sh Not tainted 5.5.0-rc4-00004-g2328d01dbd85 #105 [ 51.725078] Hardware name: STM32 (Device Tree Support) [ 51.730206] PC is at 0x90195958 [ 51.733329] LR is at 0x901c4df3 [ 51.736471] pc : [<90195958>] lr : [<901c4df3>] psr: 21000000 [ 51.742713] sp : 901e5a58 ip : 00000000 fp : 901d89fc [ 51.747911] r10: 00000000 r9 : 00000000 r8 : 00000001 [ 51.753143] r7 : 000000a8 r6 : 901e5a58 r5 : 901e5b08 r4 : ffffffff [ 51.759643] r3 : 000000a8 r2 : ffffffff r1 : 00000001 r0 : 00000001 [ 51.766122] xPSR: 21000000 [ 51.768866] CPU: 0 PID: 37 Comm: sh Not tainted 5.5.0-rc4-00004-g2328d01dbd85 #105 [ 51.776369] Hardware name: STM32 (Device Tree Support) [ 51.781594] [<0800c0c9>] (unwind_backtrace) from [<0800b25b>] (show_stack+0xb/0xc) [ 51.789166] [<0800b25b>] (show_stack) from [<0800b9eb>] (__invalid_entry+0x4b/0x4c)It is a Usage Fault happening while in thread(user) mode. PC & LR in the dump is strange in the sense that they do not point to text section.