x86/fpu: Make fpu_lock() RT compliant
cover x86/fpu: Make fpu_lock() RT compliant
FPU serialization uses
preempt_disable(); local_bh_disable();
which is silly because local_bh_disable() implicitly disables preemption. So this can be reduced to local_bh_disable().
But this does not work on RT kernels because local_bh_disable() only serializes bottom half related processing via a local lock, but does not disable preemption.
On RT kernels preempt_disable() is the right choice because on RT bottom half processing is always in thread context, so preempt_disable() implicitly protects against bottom half processing there.
Thanks,
tglx
PATCH 1/2 x86/fpu: Simplify fpregs_[un]lock()
There is no point in disabling preemption and then disabling bottom halfs.
Just disabling bottom halfs is sufficient as it implicitly disables preemption on !RT kernels.
--- arch/x86/include/asm/fpu/api.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +3 -2
@@ -29,17 +29,18 @@ extern void fpregs_mark_activate(void);* A context switch will (and softirq might) save CPU's FPU registers to * fpu->state and set TIF_NEED_FPU_LOAD leaving CPU's FPU registers in * a random state. +* +* local_bh_disable() protects against both preemption and soft interrupts +* on !RT kernels. */ static inline void fpregs_lock (void ) { -preempt_disable ();local_bh_disable (); }static inline void fpregs_unlock (void ) {local_bh_enable (); -preempt_enable (); }#ifdef CONFIG_X86_DEBUG_FPU
[tip: x86/fpu] x86/fpu: Simplify fpregs_[un]lock()
The following commit has been merged into the x86/fpu branch of tip:
The following commit has been merged into the x86/fpu branch of tip:
Commit-ID: 5f0c71278d6848b4809f83af90f28196e1505ab1 Gitweb: https://git.kernel.org/tip/5f0c71278d6848b4809f83af90f28196e1505ab1 Author: Thomas Gleixner <[email protected]> AuthorDate: Tue, 27 Oct 2020 11:09:50 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 11 Nov 2020 14:35:16 +01:00
x86/fpu: Simplify fpregs_[un]lock()
There is no point in disabling preemption and then disabling bottom halfs.
Just disabling bottom halfs is sufficient as it implicitly disables preemption on !RT kernels.
--- arch/x86/include/asm/fpu/api.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +3 -2
@@ -29,17 +29,18 @@ extern void fpregs_mark_activate(void);* A context switch will (and softirq might) save CPU's FPU registers to * fpu->state and set TIF_NEED_FPU_LOAD leaving CPU's FPU registers in * a random state. +* +* local_bh_disable() protects against both preemption and soft interrupts +* on !RT kernels. */ static inline void fpregs_lock (void ) { -preempt_disable ();local_bh_disable (); }static inline void fpregs_unlock (void ) {local_bh_enable (); -preempt_enable (); }#ifdef CONFIG_X86_DEBUG_FPU
PATCH 2/2 x86/fpu: Make kernel FPU protection RT friendly
In reply to Thomas GleixnerNon RT kernels need to protect FPU against preemption and bottom half processing. This is achieved by disabling bottom halfs via local_bh_disable() which implictly disables preemption.
On RT kernels this protection mechanism is not sufficient because local_bh_disable() does not disable preemption. It serializes bottom half related processing via a CPU local lock.
As bottom halfs are running always in thread context on RT kernels disabling preemption is the proper choice as it implicitly prevents bottom half processing.
--- arch/x86/include/asm/fpu/api.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +14 -2
@@ -32,15 +32,27 @@ extern void fpregs_mark_activate(void);* * local_bh_disable() protects against both preemption and soft interrupts * on !RT kernels. +* +* On RT kernels local_bh_disable() is not sufficient because it only +* serializes soft interrupt related sections via a local lock, but stays +* preemptible. Disabling preemption is the right choice here as bottom +* half processing is always in thread context on RT kernels so it +* implicitly prevents bottom half processing as well. */ static inline void fpregs_lock (void ) { -local_bh_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_bh_disable (); +else +preempt_disable (); }static inline void fpregs_unlock (void ) { -local_bh_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_bh_enable (); +else +preempt_enable (); }#ifdef CONFIG_X86_DEBUG_FPU
Re: [patch 2/2] x86/fpu: Make kernel FPU protection RT friendly
arch/x86/include/asm/fpu/api.h extern void fpregs_mark_activate(void); · line 40
Quoted 12 lines · Thomas Gleixner
arch/x86/include/asm/fpu/api.h +6
-0@@ -32,15 +32,27 @@ extern void fpregs_mark_activate(void);* * local_bh_disable() protects against both preemption and soft interrupts * on !RT kernels. +* +* On RT kernels local_bh_disable() is not sufficient because it only +* serializes soft interrupt related sections via a local lock, but stays +* preemptible. Disabling preemption is the right choice here as bottom +* half processing is always in thread context on RT kernels so it +* implicitly prevents bottom half processing as well.
The important part is that kernel_fpu_begin() also disables preemption and it may run in softirq. It does not use fpregs_lock() and fpregs_lock() serializes against kernel_fpu_begin().
arch/x86/include/asm/fpu/api.h extern void fpregs_mark_activate(void); · line 47
Quoted 8 lines
arch/x86/include/asm/fpu/api.h +4
-1*/ static inline void fpregs_lock (void ) { -local_bh_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_bh_disable (); +else +preempt_disable ();
Could you please swap that to
if (IS_ENABLED(CONFIG_PREEMPT_RT)) preempt_disable(); else local_bh_disable();
arch/x86/include/asm/fpu/api.h extern void fpregs_mark_activate(void); · line 49
Quoted 2 lines
arch/x86/include/asm/fpu/api.h +0
-0}
Sebastian
[tip: x86/fpu] x86/fpu: Make kernel FPU protection RT friendly
In reply to Thomas GleixnerThe following commit has been merged into the x86/fpu branch of tip:
The following commit has been merged into the x86/fpu branch of tip:
Commit-ID: cba08c5dc6dc1a906a0b5ddac9a9ac6c9a64f2e8 Gitweb: https://git.kernel.org/tip/cba08c5dc6dc1a906a0b5ddac9a9ac6c9a64f2e8 Author: Thomas Gleixner <[email protected]> AuthorDate: Tue, 27 Oct 2020 11:09:51 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 11 Nov 2020 14:35:16 +01:00
x86/fpu: Make kernel FPU protection RT friendly
Non RT kernels need to protect FPU against preemption and bottom half processing. This is achieved by disabling bottom halfs via local_bh_disable() which implictly disables preemption.
On RT kernels this protection mechanism is not sufficient because local_bh_disable() does not disable preemption. It serializes bottom half related processing via a CPU local lock.
As bottom halfs are running always in thread context on RT kernels disabling preemption is the proper choice as it implicitly prevents bottom half processing.
--- arch/x86/include/asm/fpu/api.h | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
arch/x86/include/asm/fpu/api.h +16 -2
@@ -32,15 +32,29 @@ extern void fpregs_mark_activate(void);* * local_bh_disable() protects against both preemption and soft interrupts * on !RT kernels. +* +* On RT kernels local_bh_disable() is not sufficient because it only +* serializes soft interrupt related sections via a local lock, but stays +* preemptible. Disabling preemption is the right choice here as bottom +* half processing is always in thread context on RT kernels so it +* implicitly prevents bottom half processing as well. +* +* Disabling preemption also serializes against kernel_fpu_begin(). */ static inline void fpregs_lock (void ) { -local_bh_disable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_bh_disable (); +else +preempt_disable (); }static inline void fpregs_unlock (void ) { -local_bh_enable (); +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +local_bh_enable (); +else +preempt_enable (); }#ifdef CONFIG_X86_DEBUG_FPU