Skip to contentSkip to conversation

Backport PREEMPT-RT support for ARM 32-bit

PATCH Applied Jan Kiszka 2 participants 9 messages Nov 9 – Nov 13

Subsystems (2)
lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

v2 cover Backport PREEMPT-RT support for ARM 32-bit

As a full backport via stable was rejected, this is targeting CIP 6.12 with the required changes from upstream to enable PREEMPT-RT on ARM 32-bit.

The series was derived from the ARM-specific backlog in v6.12-rt-rebase, only reconstructing it from the corresponding upstream commits - maybe worth to consider for stable-rt as well.

Jan

Sebastian Andrzej Siewior (5):
ARM: 9423/1: vfp: Provide vfp_state_hold() for VFP locking.
ARM: 9424/1: vfp: Use vfp_state_hold() in vfp_sync_hwstate().
ARM: 9425/1: vfp: Use vfp_state_hold() in vfp_support_entry().
ARM: 9426/1: vfp: Move sending signals outside of vfp_state_hold()ed
section.
ARM: 9463/1: Allow to enable RT

Thomas Gleixner (1):
ARM: 9459/1: Disable jump-label on PREEMPT_RT

 arch/arm/Kconfig         |  3 +-
 arch/arm/vfp/vfpmodule.c | 74 ++++++++++++++++++++++++++++------------
 2 files changed, 55 insertions(+), 22 deletions(-)
Signature · 1 line
2.47.3
author
#1 lore raw

PATCH v2 1/6 ARM: 9459/1: Disable jump-label on PREEMPT_RT

From: Thomas Gleixner <[email protected]>

[ Upstream commit 256d97d3587b441630c345ab3d773a9e7dcd964b ]

jump-labels are used to efficiently switch between two possible code paths. To achieve this, stop_machine() is used to keep the CPU in a known state while the opcode is modified. The usage of stop_machine() here leads to large latency spikes which can be observed on PREEMPT_RT.

Jump labels may change the target during runtime and are not restricted to debug or "configuration/ setup" part of a PREEMPT_RT system where high latencies could be defined as acceptable.

On 64-bit Arm, it is possible to use jump labels without the stop_machine() call, which architecturally provides a way to atomically change one 32-bit instruction word while keeping maintaining consistency, but this is not generally the case on 32-bit, in particular in thumb2 mode.

Disable jump-label support on a PREEMPT_RT system when SMP is enabled.

[bigeasy: Patch description.]
[arnd: add !SMP case, extend changelog]

---
 arch/arm/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm/Kconfig +1 -1
index 51c6b2db9eadf..5f0e12a84eaa1 100644
@@ -79,7 +79,7 @@ config ARM
 	select HAS_IOPORT
 	select HAVE_ARCH_AUDITSYSCALL if AEABI && !OABI_COMPAT
 	select HAVE_ARCH_BITREVERSE if (CPU_32v7M || CPU_32v7) && !CPU_32v6
-	select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU
+	select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL && !CPU_ENDIAN_BE32 && MMU && (!PREEMPT_RT || !SMP)
 	select HAVE_ARCH_KFENCE if MMU && !XIP_KERNEL
 	select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU
 	select HAVE_ARCH_KASAN if MMU && !XIP_KERNEL
Signature · 1 line
2.47.3
author
#2 lore raw

PATCH v2 2/6 ARM: 9423/1: vfp: Provide vfp_state_hold() for VFP locking.

In reply to Jan Kiszka

From: Sebastian Andrzej Siewior <[email protected]>

[ Upstream commit f26bdbe1fa9f79fa8cb0d0bf39303c3573c60552 ]

kernel_neon_begin() uses local_bh_disable() to ensure exclusive access to the VFP unit. This is broken on PREEMPT_RT because a BH disabled section remains preemptible on PREEMPT_RT.

Introduce vfp_state_hold() which uses local_bh_disable() and preempt_disable() on PREEMPT_RT. Since softirqs are processed always in thread context, disabling preemption is enough to ensure that the current context won't get interrupted by something that is using the VFP. Use it in kernel_neon_begin().

---
 arch/arm/vfp/vfpmodule.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)
arch/arm/vfp/vfpmodule.c +30 -2
index b68efe643a12c..63de164c7fc7b 100644
@@ -55,6 +55,34 @@ extern unsigned int VFP_arch_feroceon __alias(VFP_arch);
  */
 union vfp_state *vfp_current_hw_state[NR_CPUS];
 
+/*
+ * Claim ownership of the VFP unit.
+ *
+ * The caller may change VFP registers until vfp_state_release() is called.
+ *
+ * local_bh_disable() is used to disable preemption and to disable VFP
+ * processing in softirq context. On PREEMPT_RT kernels local_bh_disable() is
+ * not sufficient because it only serializes soft interrupt related sections
+ * via a local lock, but stays preemptible. Disabling preemption is the right
+ * choice here as bottom half processing is always in thread context on RT
+ * kernels so it implicitly prevents bottom half processing as well.
+ */
+static void vfp_state_hold(void)
+{
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_disable();
+	else
+		preempt_disable();
+}
+
+static void vfp_state_release(void)
+{
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_enable();
+	else
+		preempt_enable();
+}
+
 /*
  * Is 'thread's most up to date state stored in this CPUs hardware?
  * Must be called from non-preemptible context.
@@ -837,7 +865,7 @@ void kernel_neon_begin(void)
 	unsigned int cpu;
 	u32 fpexc;
 
-	local_bh_disable();
+	vfp_state_hold();
 
 	/*
 	 * Kernel mode NEON is only allowed outside of hardirq context with
@@ -868,7 +896,7 @@ void kernel_neon_end(void)
 {
 	/* Disable the NEON/VFP unit. */
 	fmxr(FPEXC, fmrx(FPEXC) & ~FPEXC_EN);
-	local_bh_enable();
+	vfp_state_release();
 }
 EXPORT_SYMBOL(kernel_neon_end);
 
Signature · 1 line
2.47.3
author
#3 lore raw

PATCH v2 3/6 ARM: 9424/1: vfp: Use vfp_state_hold() in vfp_sync_hwstate().

In reply to Jan Kiszka

From: Sebastian Andrzej Siewior <[email protected]>

[ Upstream commit b54cdbad4a39bb3abc85ac151f4882b3d92c5d79 ]

vfp_sync_hwstate() uses preempt_disable() followed by local_bh_disable() to ensure that it won't get interrupted while checking the VFP state. This harms PREEMPT_RT because softirq handling can get preempted and local_bh_disable() synchronizes the related section with a sleeping lock which does not work with disabled preemption.

Use the vfp_state_hold() to synchronize the access.

---
 arch/arm/vfp/vfpmodule.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)
arch/arm/vfp/vfpmodule.c +3 -6
index 63de164c7fc7b..1726db0500760 100644
@@ -540,11 +540,9 @@ static inline void vfp_pm_init(void) { }
  */
 void vfp_sync_hwstate(struct thread_info *thread)
 {
-	unsigned int cpu = get_cpu();
-
-	local_bh_disable();
+	vfp_state_hold();
 
-	if (vfp_state_in_hw(cpu, thread)) {
+	if (vfp_state_in_hw(raw_smp_processor_id(), thread)) {
 		u32 fpexc = fmrx(FPEXC);
 
 		/*
@@ -555,8 +553,7 @@ void vfp_sync_hwstate(struct thread_info *thread)
 		fmxr(FPEXC, fpexc);
 	}
 
-	local_bh_enable();
-	put_cpu();
+	vfp_state_release();
 }
 
 /* Ensure that the thread reloads the hardware VFP state on the next use. */
Signature · 1 line
2.47.3
author
#4 lore raw

PATCH v2 4/6 ARM: 9425/1: vfp: Use vfp_state_hold() in vfp_support_entry().

In reply to Jan Kiszka

From: Sebastian Andrzej Siewior <[email protected]>

[ Upstream commit 27035c23bad54ed552c6741210dd1c4fa50cb386 ]

vfp_entry() is invoked from exception handler and is fully preemptible. It uses local_bh_disable() to remain uninterrupted while checking the VFP state.
This is not working on PREEMPT_RT because local_bh_disable() synchronizes the relevant section but the context remains fully preemptible.

Use vfp_state_hold() for uninterrupted access.

---
 arch/arm/vfp/vfpmodule.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/arm/vfp/vfpmodule.c +3 -3
index 1726db0500760..28dafae7758b8 100644
@@ -708,7 +708,7 @@ static int vfp_support_entry(struct pt_regs *regs, u32 trigger)
 	if (!user_mode(regs))
 		return vfp_kmode_exception(regs, trigger);
 
-	local_bh_disable();
+	vfp_state_hold();
 	fpexc = fmrx(FPEXC);
 
 	/*
@@ -787,7 +787,7 @@ static int vfp_support_entry(struct pt_regs *regs, u32 trigger)
 			if (!(fpscr & FPSCR_IXE)) {
 				if (!(fpscr & FPSCR_LENGTH_MASK)) {
 					pr_debug("not VFP\n");
-					local_bh_enable();
+					vfp_state_release();
 					return -ENOEXEC;
 				}
 				fpexc |= FPEXC_DEX;
@@ -797,7 +797,7 @@ bounce:		regs->ARM_pc += 4;
 		VFP_bounce(trigger, fpexc, regs);
 	}
 
-	local_bh_enable();
+	vfp_state_release();
 	return 0;
 }
 
Signature · 1 line
2.47.3
author
#5 lore raw

PATCH v2 5/6 ARM: 9426/1: vfp: Move sending signals outside of vfp_state_hold()ed section.

In reply to Jan Kiszka

From: Sebastian Andrzej Siewior <[email protected]>

[ Upstream commit c0b5195bad63b18022f2776372fccd3ae9177705 ]

VFP_bounce() is invoked from within vfp_support_entry() and may send a signal. Sending a signal uses spinlock_t which becomes a sleeping lock on PREEMPT_RT and must not be acquired within a preempt-disabled section.

Move the vfp_raise_sigfpe() block outside of the vfp_state_hold() section.

---
 arch/arm/vfp/vfpmodule.c | 29 ++++++++++++++++++-----------
 1 file changed, 18 insertions(+), 11 deletions(-)
arch/arm/vfp/vfpmodule.c +18 -11
index 28dafae7758b8..d44867fc0c5ee 100644
@@ -268,7 +268,7 @@ static void vfp_panic(char *reason, u32 inst)
 /*
  * Process bitmask of exception conditions.
  */
-static void vfp_raise_exceptions(u32 exceptions, u32 inst, u32 fpscr, struct pt_regs *regs)
+static int vfp_raise_exceptions(u32 exceptions, u32 inst, u32 fpscr)
 {
 	int si_code = 0;
 
@@ -276,8 +276,7 @@ static void vfp_raise_exceptions(u32 exceptions, u32 inst, u32 fpscr, struct pt_
 
 	if (exceptions == VFP_EXCEPTION_ERROR) {
 		vfp_panic("unhandled bounce", inst);
-		vfp_raise_sigfpe(FPE_FLTINV, regs);
-		return;
+		return FPE_FLTINV;
 	}
 
 	/*
@@ -305,8 +304,7 @@ static void vfp_raise_exceptions(u32 exceptions, u32 inst, u32 fpscr, struct pt_
 	RAISE(FPSCR_OFC, FPSCR_OFE, FPE_FLTOVF);
 	RAISE(FPSCR_IOC, FPSCR_IOE, FPE_FLTINV);
 
-	if (si_code)
-		vfp_raise_sigfpe(si_code, regs);
+	return si_code;
 }
 
 /*
@@ -352,6 +350,8 @@ static u32 vfp_emulate_instruction(u32 inst, u32 fpscr, struct pt_regs *regs)
 static void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs)
 {
 	u32 fpscr, orig_fpscr, fpsid, exceptions;
+	int si_code2 = 0;
+	int si_code = 0;
 
 	pr_debug("VFP: bounce: trigger %08x fpexc %08x\n", trigger, fpexc);
 
@@ -397,8 +397,8 @@ static void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs)
 		 * unallocated VFP instruction but with FPSCR.IXE set and not
 		 * on VFP subarch 1.
 		 */
-		 vfp_raise_exceptions(VFP_EXCEPTION_ERROR, trigger, fpscr, regs);
-		return;
+		si_code = vfp_raise_exceptions(VFP_EXCEPTION_ERROR, trigger, fpscr);
+		goto exit;
 	}
 
 	/*
@@ -422,14 +422,14 @@ static void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs)
 	 */
 	exceptions = vfp_emulate_instruction(trigger, fpscr, regs);
 	if (exceptions)
-		vfp_raise_exceptions(exceptions, trigger, orig_fpscr, regs);
+		si_code2 = vfp_raise_exceptions(exceptions, trigger, orig_fpscr);
 
 	/*
 	 * If there isn't a second FP instruction, exit now. Note that
 	 * the FPEXC.FP2V bit is valid only if FPEXC.EX is 1.
 	 */
 	if ((fpexc & (FPEXC_EX | FPEXC_FP2V)) != (FPEXC_EX | FPEXC_FP2V))
-		return;
+		goto exit;
 
 	/*
 	 * The barrier() here prevents fpinst2 being read
@@ -441,7 +441,13 @@ static void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs)
  emulate:
 	exceptions = vfp_emulate_instruction(trigger, orig_fpscr, regs);
 	if (exceptions)
-		vfp_raise_exceptions(exceptions, trigger, orig_fpscr, regs);
+		si_code = vfp_raise_exceptions(exceptions, trigger, orig_fpscr);
+exit:
+	vfp_state_release();
+	if (si_code2)
+		vfp_raise_sigfpe(si_code2, regs);
+	if (si_code)
+		vfp_raise_sigfpe(si_code, regs);
 }
 
 static void vfp_enable(void *unused)
@@ -773,6 +779,7 @@ static int vfp_support_entry(struct pt_regs *regs, u32 trigger)
 		 * replay the instruction that trapped.
 		 */
 		fmxr(FPEXC, fpexc);
+		vfp_state_release();
 	} else {
 		/* Check for synchronous or asynchronous exceptions */
 		if (!(fpexc & (FPEXC_EX | FPEXC_DEX))) {
@@ -794,10 +801,10 @@ static int vfp_support_entry(struct pt_regs *regs, u32 trigger)
 			}
 		}
 bounce:		regs->ARM_pc += 4;
+		/* VFP_bounce() will invoke vfp_state_release() */
 		VFP_bounce(trigger, fpexc, regs);
 	}
 
-	vfp_state_release();
 	return 0;
 }
 
Signature · 1 line
2.47.3
author
#6 lore raw

PATCH v2 6/6 ARM: 9463/1: Allow to enable RT

In reply to Jan Kiszka

From: Sebastian Andrzej Siewior <[email protected]>

[ Upstream commit c6e61c06d6061750597e79c598acb5dead44c35b ]

All known issues have been adressed.
Allow to select RT.

---
 arch/arm/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/arm/Kconfig +1 -0
index 5f0e12a84eaa1..bbf84b0ac0b50 100644
@@ -40,6 +40,7 @@ config ARM
 	select ARCH_SUPPORTS_CFI_CLANG
 	select ARCH_SUPPORTS_HUGETLBFS if ARM_LPAE
 	select ARCH_SUPPORTS_PER_VMA_LOCK
+	select ARCH_SUPPORTS_RT
 	select ARCH_USE_BUILTIN_BSWAP
 	select ARCH_USE_CMPXCHG_LOCKREF
 	select ARCH_USE_MEMTEST
Signature · 1 line
2.47.3
Reviewed-by on the v2 series
#7 lore raw
In reply to Jan Kiszka

Hi!

Quoted 7 lines

As a full backport via stable was rejected, this is targeting CIP 6.12 with the required changes from upstream to enable PREEMPT-RT on ARM 32-bit.

The series was derived from the ARM-specific backlog in v6.12-rt-rebase, only reconstructing it from the corresponding upstream commits - maybe worth to consider for stable-rt as well.

Series looks good to me.

I can apply the series if it passes testing and there are no other comments.

Best regards,
Pavel

Applied
#8 lore raw
In reply to Jan Kiszka

Hi!

Quoted 7 lines

As a full backport via stable was rejected, this is targeting CIP 6.12 with the required changes from upstream to enable PREEMPT-RT on ARM 32-bit.

The series was derived from the ARM-specific backlog in v6.12-rt-rebase, only reconstructing it from the corresponding upstream commits - maybe worth to consider for stable-rt as well.

Thank you, applied. It might be good to merge relevant configs so that we start testing rt on arm32.

Best regards,
Pavel