xen/locking: fix and enhance lock debugging
cover xen/locking: fix and enhance lock debugging
This small series fixes two issues with spinlock debug code and adds lock debug code to rwlocks in order to catch IRQ violations.
Juergen Gross (2):
xen/spinlocks: spin_trylock with interrupts off is always fine xen/rwlock: add check_lock() handling to rwlocks
xen/common/spinlock.c | 17 ++++++++++------- xen/include/xen/rwlock.h | 14 ++++++++++++++ xen/include/xen/spinlock.h | 2 ++ 3 files changed, 26 insertions(+), 7 deletions(-)
Signature · 1 line
2.26.2
PATCH 1/2 xen/spinlocks: spin_trylock with interrupts off is always fine
Even if a spinlock was taken with interrupts on before calling spin_trylock() with interrupts off is fine, as it can't block.
Add a bool parameter "try" to check_lock() for handling this case.
Remove the call of check_lock() from _spin_is_locked(), as it really serves no purpose and it can even lead to false crashes, e.g. when a lock was taken correctly with interrupts enabled and the call of _spin_is_locked() happened with interrupts off. In case the lock is taken with wrong interrupt flags this will be catched when taking the lock.
--- xen/common/spinlock.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
xen/common/spinlock.c +11 -7
@@ -13,7 +13,7 @@static atomic_t spin_debug__read_mostly =ATOMIC_INIT (0 ); -static void check_lock (union lock_debug *debug) +static void check_lock (union lock_debug *debug,bool try) {bool irq_safe = !local_irq_is_enabled ();
@@ -42,7 +42,13 @@ static void check_lock(union lock_debug *debug)* * To guard against this subtle bug we latch the IRQ safety of every * spinlock in the system, on first use. +* +* A spin_trylock() or spin_is_locked() with interrupts off is always +* fine, as those can't block and above deadlock scenario doesn't apply. */ +if ( try && irq_safe ) +return ; +if (unlikely (debug->irq_safe != irq_safe) ) {union lock_debug seen, new = {0 };
@@ -102,7 +108,7 @@ void spin_debug_disable(void)#else /* CONFIG_DEBUG_LOCKS */ -#define check_lock (l) ((void)0) +#define check_lock (l, t) ((void)0)#define check_barrier (l) ((void)0)#define got_lock (l) ((void)0)#define rel_lock (l) ((void)0)
@@ -159,7 +165,7 @@ void inline _spin_lock_cb(spinlock_t *lock, void (*cb)(void *), void *data)spinlock_tickets_t tickets =SPINLOCK_TICKET_INC ;LOCK_PROFILE_VAR ; -check_lock (&lock->debug); +check_lock (&lock->debug, false);preempt_disable (); tickets.head_tail =arch_fetch_and_add (&lock->tickets.head_tail, tickets.head_tail);
@@ -220,8 +226,6 @@ void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)int _spin_is_locked (spinlock_t *lock) { -check_lock (&lock->debug); -/* * Recursive locks may be locked by another CPU, yet we return * "false" here, making this function suitable only for use in
@@ -236,7 +240,7 @@ int _spin_trylock(spinlock_t *lock) {spinlock_tickets_t old, new; -check_lock (&lock->debug); +check_lock (&lock->debug, true); old =observe_lock (&lock->tickets);if ( old.head != old.tail )return 0 ;
@@ -294,7 +298,7 @@ int _spin_trylock_recursive(spinlock_t *lock)BUILD_BUG_ON (NR_CPUS >SPINLOCK_NO_CPU );BUILD_BUG_ON (SPINLOCK_RECURSE_BITS <3 ); -check_lock (&lock->debug); +check_lock (&lock->debug, true);if (likely (lock->recurse_cpu != cpu) ) {
Signature · 1 line
2.26.2
Re: [PATCH 1/2] xen/spinlocks: spin_trylock with interrupts off is always fine
Quoted 5 lines · Juergen Gross
Even if a spinlock was taken with interrupts on before calling
spin_trylock()with interrupts off is fine, as it can't block.Add a bool parameter "try" to
check_lock()for handling this case.
Quoted 8 lines
Remove the call of
check_lock()from_spin_is_locked(), as it really serves no purpose and it can even lead to false crashes, e.g. when a lock was taken correctly with interrupts enabled and the call of_spin_is_locked()happened with interrupts off. In case the lock is taken with wrong interrupt flags this will be catched when taking the lock.Signed-off-by Juergen Gross <[email protected]>
albeit I guess ...
xen/common/spinlock.c static void check_lock(union lock_debug *debug) · line 50
Quoted 10 lines
xen/common/spinlock.c +5
-0@@ -42,7 +42,13 @@ static void check_lock(union lock_debug *debug)* * To guard against this subtle bug we latch the IRQ safety of every * spinlock in the system, on first use. +* +* A spin_trylock() or spin_is_locked() with interrupts off is always +* fine, as those can't block and above deadlock scenario doesn't apply. */ +if ( try && irq_safe ) +return ;
... the reference to spin_is_locked() here wants dropping, since ...
xen/common/spinlock.c void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags) · line 223
Quoted 5 lines
xen/common/spinlock.c +0
-1@@ -220,8 +226,6 @@ void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)int _spin_is_locked (spinlock_t *lock) { -check_lock (&lock->debug);
... you drop the call here?
Jan
Re: [PATCH 1/2] xen/spinlocks: spin_trylock with interrupts off is always fine
Quoted 32 lines · Jan Beulich
Quoted 5 lines · Juergen Gross
Even if a spinlock was taken with interrupts on before calling
spin_trylock()with interrupts off is fine, as it can't block.Add a bool parameter "try" to
check_lock()for handling this case.Quoted 8 lines
Remove the call of
check_lock()from_spin_is_locked(), as it really serves no purpose and it can even lead to false crashes, e.g. when a lock was taken correctly with interrupts enabled and the call of_spin_is_locked()happened with interrupts off. In case the lock is taken with wrong interrupt flags this will be catched when taking the lock.Signed-off-by Juergen Gross <[email protected]>Reviewed-by Jan Beulich <[email protected]>albeit I guess ...
xen/common/spinlock.c static void check_lock(union lock_debug *debug) · line 50
Quoted 10 lines
xen/common/spinlock.c +5
-0@@ -42,7 +42,13 @@ static void check_lock(union lock_debug *debug)* * To guard against this subtle bug we latch the IRQ safety of every * spinlock in the system, on first use. +* +* A spin_trylock() or spin_is_locked() with interrupts off is always +* fine, as those can't block and above deadlock scenario doesn't apply. */ +if ( try && irq_safe ) +return ;... the reference to
spin_is_locked()here wants dropping, since ...
Quoted 7 lines
xen/common/spinlock.c void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags) · line 223
Quoted 5 lines
xen/common/spinlock.c +0
-1@@ -220,8 +226,6 @@ void _spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)int _spin_is_locked (spinlock_t *lock) { -check_lock (&lock->debug);... you drop the call here?
Oh yes, this was a late modification and I didn't adapt the comment accordingly. Thanks for spotting it.
Juergen
PATCH 2/2 xen/rwlock: add check_lock() handling to rwlocks
In reply to Juergen GrossChecking whether a lock is consistently used regarding interrupts on or off is beneficial for rwlocks, too.
So add check_lock() calls to rwlock functions. For this purpose make check_lock() globally accessible.
--- xen/common/spinlock.c | 3 +-- xen/include/xen/rwlock.h | 14 ++++++++++++++ xen/include/xen/spinlock.h | 2 ++ 3 files changed, 17 insertions(+), 2 deletions(-)
xen/common/spinlock.c +1 -2
@@ -13,7 +13,7 @@static atomic_t spin_debug__read_mostly =ATOMIC_INIT (0 ); -static void check_lock (union lock_debug *debug,bool try) +void check_lock (union lock_debug *debug,bool try) {bool irq_safe = !local_irq_is_enabled ();
@@ -108,7 +108,6 @@ void spin_debug_disable(void)#else /* CONFIG_DEBUG_LOCKS */ -#define check_lock (l, t) ((void)0)#define check_barrier (l) ((void)0)#define got_lock (l) ((void)0)#define rel_lock (l) ((void)0)
xen/include/xen/rwlock.h +14 -0
@@ -65,7 +65,11 @@ static inline int _read_trylock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (likely (_can_read_lock (cnts)) ) + { +check_lock (&lock->lock.debug, true);return 1 ; + } +atomic_sub (_QR_BIAS, &lock->cnts); }preempt_enable ();
@@ -87,7 +91,10 @@ static inline void _read_lock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (likely (_can_read_lock (cnts)) ) + { +check_lock (&lock->lock.debug, false);return ; + }/* The slowpath will decrement the reader count, if necessary. */ queue_read_lock_slowpath (lock);
@@ -162,7 +169,10 @@ static inline void _write_lock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (atomic_cmpxchg (&lock->cnts,0 ,_write_lock_val ()) ==0 ) + { +check_lock (&lock->lock.debug, false);return ; + }queue_write_lock_slowpath (lock);/*
@@ -205,6 +215,8 @@ static inline int _write_trylock(rwlock_t *lock)return 0 ; } +check_lock (&lock->lock.debug, true); +/* * atomic_cmpxchg() is a full barrier so no need for an * arch_lock_acquire_barrier().
@@ -328,6 +340,8 @@ static inline void _percpu_read_lock(percpu_rwlock_t **per_cpudata,/* Drop the read lock because we don't need it anymore. */ read_unlock (&percpu_rwlock->rwlock); } +else +check_lock (&percpu_rwlock->rwlock.lock.debug, false); } static inline void _percpu_read_unlock(percpu_rwlock_t **per_cpudata,
xen/include/xen/spinlock.h +2 -0
@@ -21,11 +21,13 @@ union lock_debug { }; };#define _LOCK_DEBUG { LOCK_DEBUG_INITVAL } +void check_lock (union lock_debug *debug,bool try);void spin_debug_enable (void );void spin_debug_disable (void );#else union lock_debug { };#define _LOCK_DEBUG { } +#define check_lock (l, t) ((void)0)#define spin_debug_enable () ((void)0)#define spin_debug_disable () ((void)0)#endif
Signature · 1 line
2.26.2
Re: [PATCH 2/2] xen/rwlock: add check_lock() handling to rwlocks
xen/include/xen/rwlock.h static inline int _read_trylock(rwlock_t *lock) · line 71
Quoted 10 lines · Juergen Gross
xen/include/xen/rwlock.h +3
-0@@ -65,7 +65,11 @@ static inline int _read_trylock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (likely (_can_read_lock (cnts)) ) + { +check_lock (&lock->lock.debug, true);return 1 ; + }
Why not unconditionally earlier in the function?
xen/include/xen/rwlock.h static inline void _read_lock(rwlock_t *lock) · line 100
Quoted 11 lines
xen/include/xen/rwlock.h +3
-0@@ -87,7 +91,10 @@ static inline void _read_lock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (likely (_can_read_lock (cnts)) ) + { +check_lock (&lock->lock.debug, false);return ; + }/* The slowpath will decrement the reader count, if necessary. */ queue_read_lock_slowpath (lock);
I guess doing so here and ...
xen/include/xen/rwlock.h static inline void _write_lock(rwlock_t *lock) · line 177
Quoted 10 lines
xen/include/xen/rwlock.h +3
-0@@ -162,7 +169,10 @@ static inline void _write_lock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (atomic_cmpxchg (&lock->cnts,0 ,_write_lock_val ()) ==0 ) + { +check_lock (&lock->lock.debug, false);return ; + }queue_write_lock_slowpath (lock);
... here is okay, as the slow paths have checks anyway.
xen/include/xen/rwlock.h static inline int _write_trylock(rwlock_t *lock) · line 218
Quoted 5 lines
xen/include/xen/rwlock.h +1
-0@@ -205,6 +215,8 @@ static inline int _write_trylock(rwlock_t *lock)return 0 ; } +check_lock (&lock->lock.debug, true);
But here I again think it wants moving up.
Jan
Re: [PATCH 2/2] xen/rwlock: add check_lock() handling to rwlocks
Quoted 13 lines · Jan Beulich
xen/include/xen/rwlock.h static inline int _read_trylock(rwlock_t *lock) · line 71
Quoted 10 lines · Juergen Gross
xen/include/xen/rwlock.h +3
-0@@ -65,7 +65,11 @@ static inline int _read_trylock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (likely (_can_read_lock (cnts)) ) + { +check_lock (&lock->lock.debug, true);return 1 ; + }Why not unconditionally earlier in the function?
Its trylock, so we don't want to call check_lock() without having got the lock.
Quoted 28 lines
xen/include/xen/rwlock.h static inline void _read_lock(rwlock_t *lock) · line 100
Quoted 12 lines
xen/include/xen/rwlock.h +3
-0@@ -87,7 +91,10 @@ static inline void _read_lock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (likely (_can_read_lock (cnts)) ) + { +check_lock (&lock->lock.debug, false);return ; + }/* The slowpath will decrement the reader count, if necessary. */ queue_read_lock_slowpath (lock);I guess doing so here and ...
xen/include/xen/rwlock.h static inline void _write_lock(rwlock_t *lock) · line 177
Quoted 10 lines
xen/include/xen/rwlock.h +3
-0@@ -162,7 +169,10 @@ static inline void _write_lock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if (atomic_cmpxchg (&lock->cnts,0 ,_write_lock_val ()) ==0 ) + { +check_lock (&lock->lock.debug, false);return ; + }queue_write_lock_slowpath (lock);... here is okay, as the slow paths have checks anyway.
Quoted 7 lines
xen/include/xen/rwlock.h static inline int _write_trylock(rwlock_t *lock) · line 218
Quoted 5 lines
xen/include/xen/rwlock.h +1
-0@@ -205,6 +215,8 @@ static inline int _write_trylock(rwlock_t *lock)return 0 ; } +check_lock (&lock->lock.debug, true);But here I again think it wants moving up.
No, another trylock.
Juergen
Re: [PATCH 2/2] xen/rwlock: add check_lock() handling to rwlocks
Quoted 15 lines · Jürgen Groß
Quoted 13 lines · Jan Beulich
xen/include/xen/rwlock.h static inline int _read_trylock(rwlock_t *lock) · line 71
Quoted 10 lines · Juergen Gross
xen/include/xen/rwlock.h +3
-0@@ -65,7 +65,11 @@ static inline int _read_trylock(rwlock_t *lock)* arch_lock_acquire_barrier(). */ if ( likely(_can_read_lock(cnts)) ) + { + check_lock(&lock->lock.debug, true); return 1; + }Why not unconditionally earlier in the function?
Quoted 2 lines
Its trylock, so we don't want to call
check_lock()without having got the lock.
Hmm, OTOH we do so for spinlocks, too.
So maybe its really better to move it up.
Juergen