rqspinlock: Fix order in raw_res_spin_(un)lock_irq to allow schedule
Subsystems (2)
PATCH 1/1 rqspinlock: Fix order in raw_res_spin_(un)lock_irq to allow schedule
raw_res_spin_unlock_irqrestore() calls raw_res_spin_unlock() and then restores interrupts, this means preemption is enabled when interrupts are still disabled (as part of raw_res_spin_unlock()) so this cannot trigger an actual preemption.
This is inconsistent with other spinlock implementations (raw_spin_unlock_irqrestore() and bpf_res_spin_unlock_irqrestore() itself).
Adjust the macro to ensure interrupts are enabled before enabling preemption, allowing to schedule at that point. Make the same modification in the error path of raw_res_spin_lock_irqsave().
--- include/asm-generic/rqspinlock.h | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
include/asm-generic/rqspinlock.h +11 -3
@@ -243,12 +243,20 @@ static __always_inline void res_spin_unlock(rqspinlock_t *lock) ({ \int __ret; \local_irq_save (flags); \ - __ret =raw_res_spin_lock (lock); \ -if (__ret) \ +preempt_disable (); \ + __ret =res_spin_lock (lock); \ +if (__ret) { \local_irq_restore (flags); \ +preempt_enable (); \ + } \ __ret; \ }) -#define raw_res_spin_unlock_irqrestore (lock, flags) ({ raw_res_spin_unlock(lock); local_irq_restore(flags); }) +#define raw_res_spin_unlock_irqrestore (lock, flags) \ + ({ \ + res_spin_unlock(lock); \ + local_irq_restore(flags); \ + preempt_enable(); \ + })#endif /* __ASM_GENERIC_RQSPINLOCK_H */
base-commit: e43ffb69e0438cddd72aaa30898b4dc446f664f8
Signature · 1 line
2.54.0
Quoted 8 lines · Gabriele Monaco
raw_res_spin_unlock_irqrestore()callsraw_res_spin_unlock()and then restores interrupts, this means preemption is enabled when interrupts are still disabled (as part ofraw_res_spin_unlock()) so this cannot trigger an actual preemption.
This is inconsistent with other spinlock implementations (raw_spin_unlock_irqrestore()andbpf_res_spin_unlock_irqrestore()itself).
Quoted 5 lines
Adjust the macro to ensure interrupts are enabled before enabling preemption, allowing to schedule at that point. Make the same modification in the error path of
raw_res_spin_lock_irqsave().Fixes 101acd2e78b1 ("rqspinlock: Add macros for rqspinlock usage")
Should this be Cc:[email protected] to get backported?
Did you see this cause measurable performance problems, or did you find it through inspection?
Quoted 1 line
Signed-off-by Gabriele Monaco <[email protected]>
This should probably get merged through the BPF tree, but I've added the kernel/locking maintainers to Cc as well, since I feel it's more useful to have them look at it than me.
Maybe it would be good to update (as a separate patch) the MAINTAINERS file so the locking subsystem also includes the headers currently missing:
arch/*/include/asm/*spinlock*.harch/*/include/asm/*rwlock*.hinclude/asm-generic/*spinlock*.hinclude/asm-generic/*rwlock*.h
Arnd
(full patch quoted below)
Quoted 39 lines
--- include/asm-generic/rqspinlock.h | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)include/asm-generic/rqspinlock.h +11
-3@@ -243,12 +243,20 @@ static __always_inline voidres_spin_unlock (rqspinlock_t *lock) ({ \int __ret; \local_irq_save (flags); \ - __ret =raw_res_spin_lock (lock); \ -if (__ret) \ +preempt_disable (); \ + __ret =res_spin_lock (lock); \ +if (__ret) { \local_irq_restore (flags); \ +preempt_enable (); \ + } \ __ret; \ }) -#define raw_res_spin_unlock_irqrestore (lock, flags) ({ raw_res_spin_unlock(lock); local_irq_restore(flags); }) +#define raw_res_spin_unlock_irqrestore (lock, flags) \ + ({ \ + res_spin_unlock(lock); \ + local_irq_restore(flags); \ + preempt_enable(); \ + })#endif /*
__ASM_GENERIC_RQSPINLOCK_H*/base-commit:
e43ffb69e0438cddd72aaa30898b4dc446f664f8Signature · 1 line
2.54.0
Quoted 1 line · Arnd Bergmann
Should this be Cc:[email protected] to get backported?
Not sure if the Fixes: is enough to trigger the automation, I rarely remember to Cc:[email protected] and they're usually picked.
In case I guess I'd need to re-submit the patch right?
Quoted 2 lines
Did you see this cause measurable performance problems, or did you find it through inspection?
I noticed it while debugging an ENOMEM issue in the test_maps BPF selftest on PREEMPT_RT and this was an obvious cuplrit (irq_work not scheduled during a stress run). Turns out the problem is still there after this fix though.
Quoted 4 lines
Quoted 2 lines
Signed-off-by Gabriele Monaco <[email protected]>
Thanks,
Gabriele
Quoted 57 lines
This should probably get merged through the BPF tree, but I've added the kernel/locking maintainers to Cc as well, since I feel it's more useful to have them look at it than me.
Maybe it would be good to update (as a separate patch) the
MAINTAINERSfile so the locking subsystem also includes the headers currently missing:
arch/*/include/asm/*spinlock*.harch/*/include/asm/*rwlock*.hinclude/asm-generic/*spinlock*.hinclude/asm-generic/*rwlock*.hArnd
(full patch quoted below)
Quoted 39 lines
---
include/asm-generic/rqspinlock.h | 14 +++++++++++---1 file changed, 11 insertions(+), 3 deletions(-)include/asm-generic/rqspinlock.h +11
-3@@ -243,12 +243,20 @@ static __always_inline voidres_spin_unlock (rqspinlock_t *lock) ({ \ int __ret; \ local_irq_save(flags); \ - __ret = raw_res_spin_lock(lock); \ - if (__ret) \ + preempt_disable(); \ + __ret = res_spin_lock(lock); \ + if (__ret) { \ local_irq_restore(flags); \ + preempt_enable(); \ + } \ __ret; \ }) -#define raw_res_spin_unlock_irqrestore (lock, flags) ({ raw_res_spin_unlock(lock); local_irq_restore(flags); }) +#define raw_res_spin_unlock_irqrestore (lock, flags) \ + ({ \ + res_spin_unlock(lock); \ + local_irq_restore(flags); \ + preempt_enable(); \ + })#endif /*
__ASM_GENERIC_RQSPINLOCK_H*/base-commit:
e43ffb69e0438cddd72aaa30898b4dc446f664f8Signature · 1 line
2.54.0
Arnd Bergmann
maintainerQuoted 5 lines · Gabriele Monaco
Quoted 1 line · Arnd Bergmann
Should this be Cc:[email protected] to get backported?
Not sure if the Fixes: is enough to trigger the automation, I rarely remember to Cc:[email protected] and they're usually picked.
There is always human interaction. If you just have 'Fixes', this means someone will have to look at the patch carefully and make a decision, since a lot of bugfix patches either don't apply to old kernels or don't fall under the rules for stable backports.
If the patch gets tagged Cc:, this means it is expected to be backported and needs less manual work.
Quoted 1 line
In case I guess I'd need to re-submit the patch right?
It can be added by whoever picks up the patch.
Quoted 7 lines
Quoted 2 lines
Did you see this cause measurable performance problems, or did you find it through inspection?
I noticed it while debugging an ENOMEM issue in the
test_mapsBPF selftest onPREEMPT_RTand this was an obvious cuplrit (irq_worknot scheduled during a stress run). Turns out the problem is still there after this fix though.
Ok
Arnd
Quoted 8 lines · Gabriele Monaco
Quoted 1 line · Arnd Bergmann
Should this be Cc:[email protected] to get backported?
Not sure if the Fixes: is enough to trigger the automation, I rarely remember to Cc:[email protected] and they're usually picked.
In case I guess I'd need to re-submit the patch right?
Quoted 7 lines
Quoted 2 lines
Did you see this cause measurable performance problems, or did you find it through inspection?
I noticed it while debugging an ENOMEM issue in the
test_mapsBPF selftest onPREEMPT_RTand this was an obvious cuplrit (irq_worknot scheduled during a stress run). Turns out the problem is still there after this fix though.
I would imagine this to be the least of your problems, I think there's a bunch of blockers for complete selftests passing with PREEMPT_RT support in BPF.
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Signed-off-by Gabriele Monaco <[email protected]>
The patch makes sense to me as well.
Quoted 6 lines · Kumar Kartikeya Dwivedi
Quoted 5 lines · Gabriele Monaco
Quoted 2 lines · Arnd Bergmann
Did you see this cause measurable performance problems, or did you find it through inspection?
Quoted 9 lines
Quoted 4 lines
I noticed it while debugging an ENOMEM issue in the
test_mapsBPF selftest onPREEMPT_RTand this was an obvious cuplrit (irq_worknot scheduled during a stress run). Turns out the problem is still there after this fix though.I would imagine this to be the least of your problems, I think there's a bunch of blockers for complete selftests passing with
PREEMPT_RTsupport in BPF.
Well, I'm starting to believe that too..
At the moment on well tuned machines we are only observing ENOMEM issues in the test_maps and only when it's literally hogging the allocator (preallocation is off and 100 treads do updates in parallel).
What else are you expecting to fail under PREEMPT_RT?
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Gabriele Monaco <[email protected]>The patch makes sense to me as well.
Thanks,
Gabriele
Quoted 7 lines · Gabriele Monaco
Quoted 1 line · Arnd Bergmann
Should this be Cc:[email protected] to get backported?
Not sure if the Fixes: is enough to trigger the automation, I rarely remember to Cc:[email protected] and they're usually picked.
In case I guess I'd need to re-submit the patch right?
Yes. For whatever reason the patch didn't reach the patchwork.
Please resubmit with [PATCH bpf-next] subject, so that CI can test it properly. And collect Acks.
Quoted 9 lines · Arnd Bergmann
Quoted 8 lines · Gabriele Monaco
raw_res_spin_unlock_irqrestore()callsraw_res_spin_unlock()and then restores interrupts, this means preemption is enabled when interrupts are still disabled (as part ofraw_res_spin_unlock()) so this cannot trigger an actual preemption.
This is inconsistent with other spinlock implementations (raw_spin_unlock_irqrestore()andbpf_res_spin_unlock_irqrestore()itself).
Quoted 5 lines
Quoted 5 lines
Adjust the macro to ensure interrupts are enabled before enabling preemption, allowing to schedule at that point. Make the same modification in the error path of
raw_res_spin_lock_irqsave().Fixes 101acd2e78b1 ("rqspinlock: Add macros for rqspinlock usage")
Yeah, this is right. spinlocks always get one preempt_disable, in addition they might also get irq or bh disable.
Waiman Long
Acked-byQuoted 73 lines · Arnd Bergmann
Quoted 8 lines · Gabriele Monaco
raw_res_spin_unlock_irqrestore()callsraw_res_spin_unlock()and then restores interrupts, this means preemption is enabled when interrupts are still disabled (as part ofraw_res_spin_unlock()) so this cannot trigger an actual preemption.
This is inconsistent with other spinlock implementations (raw_spin_unlock_irqrestore()andbpf_res_spin_unlock_irqrestore()itself).Quoted 5 lines
Adjust the macro to ensure interrupts are enabled before enabling preemption, allowing to schedule at that point. Make the same modification in the error path of
raw_res_spin_lock_irqsave().Fixes 101acd2e78b1 ("rqspinlock: Add macros for rqspinlock usage")Should this be Cc:[email protected] to get backported?
Did you see this cause measurable performance problems, or did you find it through inspection?
Quoted 1 line
Signed-off-by Gabriele Monaco <[email protected]>This should probably get merged through the BPF tree, but I've added the kernel/locking maintainers to Cc as well, since I feel it's more useful to have them look at it than me.
Maybe it would be good to update (as a separate patch) the
MAINTAINERSfile so the locking subsystem also includes the headers currently missing:
arch/*/include/asm/*spinlock*.harch/*/include/asm/*rwlock*.hinclude/asm-generic/*spinlock*.hinclude/asm-generic/*rwlock*.hArnd
(full patch quoted below)
Quoted 34 lines
--- include/asm-generic/rqspinlock.h | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)include/asm-generic/rqspinlock.h +11
-3@@ -243,12 +243,20 @@ static __always_inline voidres_spin_unlock (rqspinlock_t *lock) ({ \int __ret; \local_irq_save (flags); \ - __ret =raw_res_spin_lock (lock); \ -if (__ret) \ +preempt_disable (); \ + __ret =res_spin_lock (lock); \ +if (__ret) { \local_irq_restore (flags); \ +preempt_enable (); \ + } \ __ret; \ }) -#define raw_res_spin_unlock_irqrestore (lock, flags) ({ raw_res_spin_unlock(lock); local_irq_restore(flags); }) +#define raw_res_spin_unlock_irqrestore (lock, flags) \ + ({ \ + res_spin_unlock(lock); \ + local_irq_restore(flags); \ + preempt_enable(); \ + })
Quoted 5 lines
Quoted 5 lines
#endif /* __ASM_GENERIC_RQSPINLOCK_H */base-commit:
e43ffb69e0438cddd72aaa30898b4dc446f664f8Signature · 1 line
2.54.0
Disabling interrupt will also disable preemption. However, the possible side effect of scheduler preemption when preemption is enabled will make these res_spin_lock APIs behave more like regular spinlock. So