Skip to contentSkip to conversation

x86/fpu: Make fpu_lock() RT compliant

PATCH Applied Thomas Gleixner 3 participants 6 messages Oct 27, 2020 – Nov 11, 2020 x86

lore
Share & tools
lore
#0 lore raw

cover x86/fpu: Make fpu_lock() RT compliant

FPU serialization uses

    preempt_disable(); local_bh_disable();

which is silly because local_bh_disable() implicitly disables preemption. So this can be reduced to local_bh_disable().

But this does not work on RT kernels because local_bh_disable() only serializes bottom half related processing via a local lock, but does not disable preemption.

On RT kernels preempt_disable() is the right choice because on RT bottom half processing is always in thread context, so preempt_disable() implicitly protects against bottom half processing there.

Thanks,

tglx

#1 lore raw

PATCH 1/2 x86/fpu: Simplify fpregs_[un]lock()

There is no point in disabling preemption and then disabling bottom halfs.

Just disabling bottom halfs is sufficient as it implicitly disables preemption on !RT kernels.

---
 arch/x86/include/asm/fpu/api.h |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +3 -2
@@ -29,17 +29,18 @@ extern void fpregs_mark_activate(void);
  * A context switch will (and softirq might) save CPU's FPU registers to
  * fpu->state and set TIF_NEED_FPU_LOAD leaving CPU's FPU registers in
  * a random state.
+ *
+ * local_bh_disable() protects against both preemption and soft interrupts
+ * on !RT kernels.
  */
 static inline void fpregs_lock(void)
 {
-	preempt_disable();
 	local_bh_disable();
 }
 
 static inline void fpregs_unlock(void)
 {
 	local_bh_enable();
-	preempt_enable();
 }
 
 #ifdef CONFIG_X86_DEBUG_FPU
#2 lore raw

[tip: x86/fpu] x86/fpu: Simplify fpregs_[un]lock()

The following commit has been merged into the x86/fpu branch of tip:

The following commit has been merged into the x86/fpu branch of tip:

Commit-ID:     5f0c71278d6848b4809f83af90f28196e1505ab1
Gitweb:        https://git.kernel.org/tip/5f0c71278d6848b4809f83af90f28196e1505ab1
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Tue, 27 Oct 2020 11:09:50 +01:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 11 Nov 2020 14:35:16 +01:00

x86/fpu: Simplify fpregs_[un]lock()

There is no point in disabling preemption and then disabling bottom halfs.

Just disabling bottom halfs is sufficient as it implicitly disables preemption on !RT kernels.

---
 arch/x86/include/asm/fpu/api.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +3 -2
index dcd9503..2c5bef7 100644
@@ -29,17 +29,18 @@ extern void fpregs_mark_activate(void);
  * A context switch will (and softirq might) save CPU's FPU registers to
  * fpu->state and set TIF_NEED_FPU_LOAD leaving CPU's FPU registers in
  * a random state.
+ *
+ * local_bh_disable() protects against both preemption and soft interrupts
+ * on !RT kernels.
  */
 static inline void fpregs_lock(void)
 {
-	preempt_disable();
 	local_bh_disable();
 }
 
 static inline void fpregs_unlock(void)
 {
 	local_bh_enable();
-	preempt_enable();
 }
 
 #ifdef CONFIG_X86_DEBUG_FPU
#3 lore raw

PATCH 2/2 x86/fpu: Make kernel FPU protection RT friendly

In reply to Thomas Gleixner

Non RT kernels need to protect FPU against preemption and bottom half processing. This is achieved by disabling bottom halfs via local_bh_disable() which implictly disables preemption.

On RT kernels this protection mechanism is not sufficient because local_bh_disable() does not disable preemption. It serializes bottom half related processing via a CPU local lock.

As bottom halfs are running always in thread context on RT kernels disabling preemption is the proper choice as it implicitly prevents bottom half processing.

---
 arch/x86/include/asm/fpu/api.h |   16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +14 -2
@@ -32,15 +32,27 @@ extern void fpregs_mark_activate(void);
  *
  * local_bh_disable() protects against both preemption and soft interrupts
  * on !RT kernels.
+ *
+ * On RT kernels local_bh_disable() is not sufficient because it only
+ * serializes soft interrupt related sections via a local lock, but stays
+ * preemptible. Disabling preemption is the right choice here as bottom
+ * half processing is always in thread context on RT kernels so it
+ * implicitly prevents bottom half processing as well.
  */
 static inline void fpregs_lock(void)
 {
-	local_bh_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_disable();
+	else
+		preempt_disable();
 }
 
 static inline void fpregs_unlock(void)
 {
-	local_bh_enable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_enable();
+	else
+		preempt_enable();
 }
 
 #ifdef CONFIG_X86_DEBUG_FPU
#4 lore raw

Re: [patch 2/2] x86/fpu: Make kernel FPU protection RT friendly

arch/x86/include/asm/fpu/api.h extern void fpregs_mark_activate(void); · line 40

Quoted 12 lines · Thomas Gleixner
arch/x86/include/asm/fpu/api.h +6 -0
@@ -32,15 +32,27 @@ extern void fpregs_mark_activate(void);
  *
  * local_bh_disable() protects against both preemption and soft interrupts
  * on !RT kernels.
+ *
+ * On RT kernels local_bh_disable() is not sufficient because it only
+ * serializes soft interrupt related sections via a local lock, but stays
+ * preemptible. Disabling preemption is the right choice here as bottom
+ * half processing is always in thread context on RT kernels so it
+ * implicitly prevents bottom half processing as well.

The important part is that kernel_fpu_begin() also disables preemption and it may run in softirq. It does not use fpregs_lock() and fpregs_lock() serializes against kernel_fpu_begin().

arch/x86/include/asm/fpu/api.h extern void fpregs_mark_activate(void); · line 47

Quoted 8 lines
arch/x86/include/asm/fpu/api.h +4 -1
  */
 static inline void fpregs_lock(void)
 {
-	local_bh_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_disable();
+	else
+		preempt_disable();

Could you please swap that to

	if (IS_ENABLED(CONFIG_PREEMPT_RT))
		preempt_disable();
	else
		local_bh_disable();

arch/x86/include/asm/fpu/api.h extern void fpregs_mark_activate(void); · line 49

Quoted 2 lines
arch/x86/include/asm/fpu/api.h +0 -0
 }
 

Sebastian

#5 lore raw

[tip: x86/fpu] x86/fpu: Make kernel FPU protection RT friendly

In reply to Thomas Gleixner
The following commit has been merged into the x86/fpu branch of tip:

The following commit has been merged into the x86/fpu branch of tip:

Commit-ID:     cba08c5dc6dc1a906a0b5ddac9a9ac6c9a64f2e8
Gitweb:        https://git.kernel.org/tip/cba08c5dc6dc1a906a0b5ddac9a9ac6c9a64f2e8
Author:        Thomas Gleixner <[email protected]>
AuthorDate:    Tue, 27 Oct 2020 11:09:51 +01:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Wed, 11 Nov 2020 14:35:16 +01:00

x86/fpu: Make kernel FPU protection RT friendly

Non RT kernels need to protect FPU against preemption and bottom half processing. This is achieved by disabling bottom halfs via local_bh_disable() which implictly disables preemption.

On RT kernels this protection mechanism is not sufficient because local_bh_disable() does not disable preemption. It serializes bottom half related processing via a CPU local lock.

As bottom halfs are running always in thread context on RT kernels disabling preemption is the proper choice as it implicitly prevents bottom half processing.

---
 arch/x86/include/asm/fpu/api.h | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +16 -2
index 2c5bef7..a5aba4a 100644
@@ -32,15 +32,29 @@ extern void fpregs_mark_activate(void);
  *
  * local_bh_disable() protects against both preemption and soft interrupts
  * on !RT kernels.
+ *
+ * On RT kernels local_bh_disable() is not sufficient because it only
+ * serializes soft interrupt related sections via a local lock, but stays
+ * preemptible. Disabling preemption is the right choice here as bottom
+ * half processing is always in thread context on RT kernels so it
+ * implicitly prevents bottom half processing as well.
+ *
+ * Disabling preemption also serializes against kernel_fpu_begin().
  */
 static inline void fpregs_lock(void)
 {
-	local_bh_disable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_disable();
+	else
+		preempt_disable();
 }
 
 static inline void fpregs_unlock(void)
 {
-	local_bh_enable();
+	if (!IS_ENABLED(CONFIG_PREEMPT_RT))
+		local_bh_enable();
+	else
+		preempt_enable();
 }
 
 #ifdef CONFIG_X86_DEBUG_FPU