queued spinlocks (i386)
I am sorry for being completely off-topic, but I've been wondering for the long time...
What if we replace raw_spinlock_t.slock with "struct task_struct *owner" ?
void _spin_lock(spinlock_t *lock)
{
struct task_struct *owner;
for (;;) {
preempt_disable();
if (likely(_raw_spin_trylock(lock)))
break;
preempt_enable();
while (!spin_can_lock(lock)) {
rcu_read_lock();
owner = lock->owner;
if (owner && current->prio < owner->prio &&
!test_tsk_thread_flag(owner, TIF_NEED_RESCHED))
set_tsk_thread_flag(owner, TIF_NEED_RESCHED);
rcu_read_unlock();
cpu_relax();
}
}
lock->owner = current;
}
void _spin_unlock(spinlock_t *lock)
{
lock->owner = NULL;
_raw_spin_unlock(lock);
preempt_enable();
}Now we don't need need_lockbreak(lock), need_resched() is enough, and we take ->prio into consideration.
Makes sense? Or stupid?
Oleg.
Quoted 2 lines · Oleg Nesterov
Makes sense? Or stupid?
Interesting idea. Probably worth testing.
-Andi
Quoted 36 lines · Oleg Nesterov
I am sorry for being completely off-topic, but I've been wondering for the long time...
What if we replace raw_spinlock_t.slock with "struct
task_struct*owner" ?void _spin_lock(spinlock_t *lock) { struct task_struct *owner; for (;;) { preempt_disable(); if (likely(_raw_spin_trylock(lock))) break; preempt_enable(); while (!spin_can_lock(lock)) { rcu_read_lock(); owner = lock->owner; if (owner && current->prio < owner->prio && !test_tsk_thread_flag(owner, TIF_NEED_RESCHED)) set_tsk_thread_flag(owner, TIF_NEED_RESCHED); rcu_read_unlock(); cpu_relax(); } } lock->owner = current; } void _spin_unlock(spinlock_t *lock) { lock->owner = NULL; _raw_spin_unlock(lock); preempt_enable(); }
Quoted 4 lines
Now we don't need
need_lockbreak(lock),need_resched()is enough, and we take ->prio into consideration.Makes sense? Or stupid?
Well with my queued spinlocks, all that lockbreak stuff can just come out of the spin_lock, break_lock out of the spinlock structure, and need_lockbreak just becomes (lock->qhead - lock->qtail > 1).
Quoted 4 lines · Nick Piggin
Well with my queued spinlocks, all that lockbreak stuff can just come out of the
spin_lock,break_lockout of the spinlock structure, andneed_lockbreakjust becomes (lock->qhead-lock->qtail> 1).
Q: queued spinlocks are not CONFIG_PREEMPT friendly,
Review
Quoted 4 lines
(quoted diff) +4
-0+ asm volatile(LOCK_PREFIX "xaddw %0, %1\n\t" + : "+r" (pos), "+m" (lock->qhead) : : "memory"); + while (unlikely(pos != lock->qtail)) + cpu_relax();
once we incremented lock->qhead, we have no optiion but should spin with preemption disabled until pos == lock->qtail, yes?
Oleg.
Quoted 7 lines · Oleg Nesterov
Quoted 4 lines · Nick Piggin
Well with my queued spinlocks, all that lockbreak stuff can just come out of the
spin_lock,break_lockout of the spinlock structure, andneed_lockbreakjust becomes (lock->qhead-lock->qtail> 1).Q: queued spinlocks are not
CONFIG_PREEMPTfriendly,
Why? Is CONFIG_PREEMPT friendly to anyone? :)
Quoted 7 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ asm volatile(LOCK_PREFIX "xaddw %0, %1\n\t" + : "+r" (pos), "+m" (lock->qhead) : : "memory"); + while (unlikely(pos != lock->qtail)) + cpu_relax();once we incremented
lock->qhead, we have no optiion but should spin with preemption disabled until pos ==lock->qtail, yes?
Yes, preemption and deterministic spinlock policies are not friends.
- Davide
Quoted 11 lines · Davide Libenzi
Quoted 7 lines · Oleg Nesterov
Quoted 4 lines · Nick Piggin
Well with my queued spinlocks, all that lockbreak stuff can just come out of the
spin_lock,break_lockout of the spinlock structure, andneed_lockbreakjust becomes (lock->qhead-lock->qtail> 1).Q: queued spinlocks are not
CONFIG_PREEMPTfriendly,Why? Is
CONFIG_PREEMPTfriendly to anyone? :)
Until someone fixes all the places in the kernel where scheduling can be held off for tens of milliseconds, CONFIG_PREEMPT will be an absolute requirement for many applications like audio and gaming.
Many of these were fixed a while back during early -rt development but at some point the process stalled as the remaining cases were too hard to fix...
Lee
Quoted 10 lines · Lee Revell
Quoted 9 lines · Davide Libenzi
Quoted 3 lines · Oleg Nesterov
Quoted 2 lines · Nick Piggin
Well with my queued spinlocks, all that lockbreak stuff can just come
out
Quoted 3 lines
Quoted 2 lines
of the
spin_lock,break_lockout of the spinlock structure, andneed_lockbreakjust becomes (lock->qhead-lock->qtail> 1).
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Q: queued spinlocks are not
CONFIG_PREEMPTfriendly,Why? Is
CONFIG_PREEMPTfriendly to anyone? :)Until someone fixes all the places in the kernel where scheduling can be held off for tens of milliseconds,
CONFIG_PREEMPTwill be an absolute requirement for many applications like audio and gaming.
There's nothing wrong with CONFIG_PREEMPT for those users. We have a few other performance concessions activated with CONFIG_PREEMPT on. I think a usual upper of a few miliseconds (especially for SMP) is reasonable for a non preempt kernel.
Quoted 9 lines · Nick Piggin
Quoted 3 lines · Lee Revell
Until someone fixes all the places in the kernel where scheduling can be held off for tens of milliseconds,
CONFIG_PREEMPTwill be an absolute requirement for many applications like audio and gaming.There's nothing wrong with
CONFIG_PREEMPTfor those users. We have a few other performance concessions activated withCONFIG_PREEMPTon. I think a usual upper of a few miliseconds (especially for SMP) is reasonable for a non preempt kernel.
This is within reach - the only major offender left is rt_secret_rebuild (and possibly other areas of the route cache handling). Eric Dumazet had some suggestions to fix it, but the details are beyond my area of expertise.
Lee
Quoted 7 lines · Oleg Nesterov
Quoted 4 lines · Nick Piggin
Well with my queued spinlocks, all that lockbreak stuff can just come out of the
spin_lock,break_lockout of the spinlock structure, andneed_lockbreakjust becomes (lock->qhead-lock->qtail> 1).Q: queued spinlocks are not
CONFIG_PREEMPTfriendly,
I consider the re-enabling of preemption and interrupts to be a hack anyway. Because if you already have interrupts or preemption disabled at entry time, they will remain disabled.
IMO the real solution is to ensure spinlock critical sections don't get too large, and perhaps use fair spinlocks to prevent starvation.
Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +4
-0+ asm volatile(LOCK_PREFIX "xaddw %0, %1\n\t" + : "+r" (pos), "+m" (lock->qhead) : : "memory"); + while (unlikely(pos != lock->qtail)) + cpu_relax();once we incremented
lock->qhead, we have no optiion but should spin with preemption disabled until pos ==lock->qtail, yes?
Correct. For the purposes of deadlock behaviour, we have effectively taken the lock at that point.