Lock ordering documentation and annotation for lockdep
Subsystems (16)
cover Lock ordering documentation and annotation for lockdep
The first patch introduces the long asked locking documentation regarding spinlock_t vs raw_spinlock_t (and other locking primitives). Followed by updates to obey the rules including a change to completions to use struct swait_queue_head instead of wait_queue_head_t.
It ends with lockdep updates to recognize the "bad" lock nesting. This new lockdep feature is activated by CONFIG_PROVE_RAW_LOCK_NESTING. Once enabled it will report for instance printk, the memory allocator or the workqueue implementation. This is known and it is worked on.
Sebastian
PATCH 1/9 Documentation: Add lock ordering and nesting documentation
From: Thomas Gleixner <[email protected]>
The kernel provides a variety of locking primitives. The nesting of these lock types and the implications of them on RT enabled kernels is nowhere documented.
Add initial documentation.
--- Documentation/locking/index.rst | 1 + Documentation/locking/locktypes.rst | 298 ++++++++++++++++++++++++++++ 2 files changed, 299 insertions(+) create mode 100644 Documentation/locking/locktypes.rst
Documentation/locking/index.rst +1 -0
@@ -7,6 +7,7 @@ locking .. toctree:: :maxdepth: 1 + locktypes lockdep-design lockstat locktorture
Documentation/locking/locktypes.rst new file +298 -0
298 lines not on this page · raw message
Signature · 1 line
2.25.1
Randy Dunlap
reviewerRe: [PATCH 1/9] Documentation: Add lock ordering and nesting documentation
Hi,
A few comments for your consideration:
Quoted 43 lines · Sebastian Andrzej Siewior
From: Thomas Gleixner <[email protected]>
The kernel provides a variety of locking primitives. The nesting of these lock types and the implications of them on RT enabled kernels is nowhere documented.
Add initial documentation.
Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Sebastian Andrzej Siewior <[email protected]>--- Documentation/locking/index.rst | 1 + Documentation/locking/locktypes.rst | 298 ++++++++++++++++++++++++++++ 2 files changed, 299 insertions(+) create mode 100644 Documentation/locking/locktypes.rstDocumentation/locking/locktypes.rst new file +21
-0@@ -0,0 +1,298 @@ +.. _kernel_hacking_locktypes: + +========================== +Lock types and their rules +========================== + +Introduction +============ + +The kernel provides a variety of locking primitives which can be divided +into two categories: + + - Sleeping locks + - Spinning locks + +This document describes the lock types at least at the conceptual level and +provides rules for nesting of lock types also under the aspect of PREEMPT_RT. + +Lock categories +=============== +
Documentation/locking/locktypes.rst line 31
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+Sleeping locks +-------------- + +Sleeping locks can only be acquired in preemptible task context. + +Some of the implementations allow try_lock() attempts from other contexts, +but that has to be really evaluated carefully including the question +whether the unlock can be done from that context safely as well. + +Note, that some lock types change their implementation details when
Drop comma.
Quoted 25 lines
Documentation/locking/locktypes.rst new file +25
-0+debugging is enabled, so this should be really only considered if there is +no other option. + +Sleeping lock types: + + - mutex + - rt_mutex + - semaphore + - rw_semaphore + - ww_mutex + - percpu_rw_semaphore + +On a PREEMPT_RT enabled kernel the following lock types are converted to +sleeping locks: + + - spinlock_t + - rwlock_t + +Spinning locks +-------------- + + - raw_spinlock_t + - bit spinlocks + +On a non PREEMPT_RT enabled kernel the following lock types are spinning
Documentation/locking/locktypes.rst line 66
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+locks as well: + + - spinlock_t + - rwlock_t + +Spinning locks implicitly disable preemption and the lock / unlock functions +can have suffixes which apply further protections: + + =================== ==================================================== + _bh() Disable / enable bottom halfs (soft interrupts)
halves
Quoted 70 lines
Documentation/locking/locktypes.rst new file +70
-0+ _irq() Disable / enable interrupts + _irqsave/restore() Save and disable / restore interrupt disabled state + =================== ==================================================== + + +rtmutex +======= + +RT-mutexes are mutexes with support for priority inheritance (PI). + +PI has limitations on non PREEMPT_RT enabled kernels due to preemption and +interrupt disabled sections. + +On a PREEMPT_RT enabled kernel most of these sections are fully +preemptible. This is possible because PREEMPT_RT forces most executions +into task context, especially interrupt handlers and soft interrupts, which +allows to substitute spinlock_t and rwlock_t with RT-mutex based +implementations. + + +raw_spinlock_t and spinlock_t +============================= + +raw_spinlock_t +-------------- + +raw_spinlock_t is a strict spinning lock implementation regardless of the +kernel configuration including PREEMPT_RT enabled kernels. + +raw_spinlock_t is to be used only in real critical core code, low level +interrupt handling and places where protecting (hardware) state is required +to be safe against preemption and eventually interrupts. + +Another reason to use raw_spinlock_t is when the critical section is tiny +to avoid the overhead of spinlock_t on a PREEMPT_RT enabled kernel in the +contended case. + +spinlock_t +---------- + +The semantics of spinlock_t change with the state of CONFIG_PREEMPT_RT. + +On a non PREEMPT_RT enabled kernel spinlock_t is mapped to raw_spinlock_t +and has exactly the same semantics. + +spinlock_t and PREEMPT_RT +------------------------- + +On a PREEMPT_RT enabled kernel spinlock_t is mapped to a separate +implementation based on rt_mutex which changes the semantics: + + - Preemption is not disabled + + - The hard interrupt related suffixes for spin_lock / spin_unlock + operations (_irq, _irqsave / _irqrestore) do not affect the CPUs + interrupt disabled state + + - The soft interrupt related suffix (_bh()) is still disabling the + execution of soft interrupts, but contrary to a non PREEMPT_RT enabled + kernel, which utilizes the preemption count, this is achieved by a per + CPU bottom half locking mechanism. + +All other semantics of spinlock_t are preserved: + + - Migration of tasks which hold a spinlock_t is prevented. On a non + PREEMPT_RT enabled kernel this is implicit due to preemption disable. + PREEMPT_RT has a separate mechanism to achieve this. This ensures that + pointers to per CPU variables stay valid even if the task is preempted. + + - Task state preservation. The task state is not affected when a lock is
Documentation/locking/locktypes.rst line 146
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+ contended and the task has to schedule out and wait for the lock to + become available. The lock wake up restores the task state unless there + was a regular (not lock related) wake up on the task. This ensures that + the task state rules are always correct independent of the kernel + configuration. + +rwlock_t +======== + +rwlock_t is a multiple readers and single writers lock mechanism.
writer
Quoted 6 lines
Documentation/locking/locktypes.rst new file +6
-0+ +On a non PREEMPT_RT enabled kernel rwlock_t is implemented as a spinning +lock and the suffix rules of spinlock_t apply accordingly. The +implementation is fair and prevents writer starvation. + +rwlock_t and PREEMPT_RT
Documentation/locking/locktypes.rst line 162
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+----------------------- + +On a PREEMPT_RT enabled kernel rwlock_t is mapped to a separate +implementation based on rt_mutex which changes the semantics: + + - Same changes as for spinlock_t + + - The implementation is not fair and can cause writer starvation under + certain circumstances. The reason for this is that a writer cannot + inherit its priority to multiple readers. Readers which are blocked
^^^^^^^ I think this is backwards. Maybe more like so:
a writer cannot
bequeath or grant or bestow or pass down ... its priority to
Quoted 31 lines
Documentation/locking/locktypes.rst new file +31
-0+ on a writer fully support the priority inheritance protocol. + + +PREEMPT_RT caveats +================== + +spinlock_t and rwlock_t +----------------------- + +The substitution of spinlock_t and rwlock_t on PREEMPT_RT enabled kernels +with RT-mutex based implementations has a few implications. + +On a non PREEMPT_RT enabled kernel the following code construct is +perfectly fine:: + + local_irq_disable(); + spin_lock(&lock); + +and fully equivalent to:: + + spin_lock_irq(&lock); + +Same applies to rwlock_t and the _irqsave() suffix variant. + +On a PREEMPT_RT enabled kernel this breaks because the RT-mutex +substitution expects a fully preemptible context. + +The preferred solution is to use :c:func:`spin_lock_irq()` or +:c:func:`spin_lock_irqsave()` and their unlock counterparts. + +PREEMPT_RT also offers a local_lock mechanism to substitute the
Documentation/locking/locktypes.rst line 203
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+local_irq_disable/save() constructs in cases where a separation of the +interrupt disabling and the locking is really unavoidable. This should be +restricted to very rare cases. + + +raw_spinlock_t +-------------- + +As raw_spinlock_t locking disables preemption and eventually interrupts the +code inside the critical region has to be careful to avoid calls into code
Can I buy a comma in there somewhere, please?
I don't get it as is.
Quoted 12 lines
Documentation/locking/locktypes.rst new file +12
-0+which takes regular spinlock_t or rwlock_t. A prime example is memory +allocation. + +On a non PREEMPT_RT enabled kernel the following code construct is +perfectly fine code:: + + raw_spin_lock(&lock); + p = kmalloc(sizeof(*p), GFP_ATOMIC); + +On a PREEMPT_RT enabled kernel this breaks because the memory allocator is +fully preemptible and therefore does not support allocations from truly +atomic contexts.
Documentation/locking/locktypes.rst line 225
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+ +Contrary to that the following code construct is perfectly fine on +PREEMPT_RT as spin_lock() does not disable preemption:: + + spin_lock(&lock); + p = kmalloc(sizeof(*p), GFP_ATOMIC); + +Most places which use GFP_ATOMIC allocations are safe on PREEMPT_RT as the +execution is forced into thread context and the lock substitution is +ensuring preemptability.
preemptibility would go along with preemptible
Documentation/locking/locktypes.rst line 232
Quoted 7 lines
Documentation/locking/locktypes.rst new file +7
-0+ + +bit spinlocks +------------- + +Bit spinlocks are problematic for PREEMPT_RT as they cannot be easily +substituted by a RT-mutex based implementation for obvious reasons.
by an
(IMO; depends on how you pronounce it)
Quoted 15 lines
Documentation/locking/locktypes.rst new file +15
-0+ +The semantics of bit spinlocks are preserved on a PREEMPT_RT enabled kernel +and the caveats vs. raw_spinlock_t apply. + +Some bit spinlocks are substituted by regular spinlock_t for PREEMPT_RT but +this requires conditional (#ifdef'ed) code changes at the usage side while +the spinlock_t substitution is simply done by the compiler and the +conditionals are restricted to header files and core implementation of the +locking primitives and the usage sites do not require any changes. + + +Lock type nesting rules +======================= + +The most basic rules are:
Documentation/locking/locktypes.rst line 257
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+ + - Lock types of the same lock category (sleeping, spinning) can nest + arbitrarily as long as they respect the general lock ordering rules to + prevent deadlocks. + + - Sleeping lock types cannot nest inside spinning lock types. + + - Spinning lock types can nest inside sleeping lock types. + +These rules apply in general independent of CONFIG_PREEMPT_RT.
independently
Quoted 14 lines
Documentation/locking/locktypes.rst new file +14
-0+ +As PREEMPT_RT changes the lock category of spinlock_t and rwlock_t from +spinning to sleeping this has obviously restrictions how they can nest with +raw_spinlock_t. + +This results in the following nest ordering: + + 1) Sleeping locks + 2) spinlock_t and rwlock_t + 3) raw_spinlock_t and bit spinlocks + +Lockdep is aware of these constraints to ensure that they are respected. + +
Documentation/locking/locktypes.rst line 281
Quoted 10 lines
Documentation/locking/locktypes.rst new file +10
-0+Owner semantics +=============== + +Most lock types in the Linux kernel have strict owner semantics, i.e. the +context (task) which acquires a lock has to release it. + +There are two exceptions: + + - semaphores + - rwsem
rwsems
(plural, like semaphores)
Documentation/locking/locktypes.rst line 288
Quoted 7 lines
Documentation/locking/locktypes.rst new file +7
-0+ +semaphores have no strict owner semantics for historical reasons. They are +often used for both serialization and waiting purposes. That's generally +discouraged and should be replaced by separate serialization and wait +mechanisms. + +rwsem have grown interfaces which allow non owner release for special
rwsems non-owner
Documentation/locking/locktypes.rst line 298
Quoted 11 lines
Documentation/locking/locktypes.rst new file +10
-0+purposes. This usage is problematic on PREEMPT_RT because PREEMPT_RT +substitutes all locking primitives except semaphores with RT-mutex based +implementation to provide priority inheritance for all lock types except +the truly spinning ones. Priority inheritance on ownerless locks is +obviously impossible. + +For now the rwsem non-owner release excludes code which utilizes it from +being used on PREEMPT_RT enabled kernels. In same cases this can be +mitigated by disabling portions of the code, in other cases the complete +functionality has to be disabled until a workable solution has been found.
cheers.
Signature · 1 line
~Randy
Re: [PATCH 1/9] Documentation: Add lock ordering and nesting documentation
Quoted 1 line · Randy Dunlap
Hi,
Hi Randy,
Quoted 1 line
A few comments for your consideration:
I merged all of you comments but two:
Quoted 1 line · Sebastian Andrzej Siewior
…
Quoted 7 lines
Quoted 7 lines
Documentation/locking/locktypes.rst new file +7
-0+rwlock_t and PREEMPT_RT +----------------------- + +On a PREEMPT_RT enabled kernel rwlock_t is mapped to a separate +implementation based on rt_mutex which changes the semantics: + + - Same changes as for spinlock_t
Quoted 8 lines
Documentation/locking/locktypes.rst line 162
Quoted 4 lines
Documentation/locking/locktypes.rst new file +4
-0+ + - The implementation is not fair and can cause writer starvation under + certain circumstances. The reason for this is that a writer cannot + inherit its priority to multiple readers. Readers which are blocked^^^^^^^ I think this is backwards. Maybe more like so:
a writer cannot
bequeath or grant or bestow or pass down ... its priority to
So the term "inherit" is the problem. The protocol is officially called PI which is short for Priority Inheritance. Other documentation, RT-mutex for instance, is also using this term when it is referring to altering the priority of a task. For that reason I prefer to keep using this term.
Quoted 1 line
Quoted 1 line
+ on a writer fully support the priority inheritance protocol.
…
Quoted 8 lines
Documentation/locking/locktypes.rst line 203
Quoted 5 lines
Documentation/locking/locktypes.rst new file +5
-0+raw_spinlock_t +-------------- + +As raw_spinlock_t locking disables preemption and eventually interrupts the +code inside the critical region has to be careful to avoid calls into codeCan I buy a comma in there somewhere, please?
I don't get it as is.
What about
| As raw_spinlock_t locking disables preemption, and eventually interrupts, the | code inside the critical region has to be careful to avoid calls into code
any better?
…
Sebastian
Randy Dunlap
reviewerRe: [PATCH 1/9] Documentation: Add lock ordering and nesting documentation
Quoted 26 lines · Sebastian Andrzej Siewior
Quoted 1 line · Randy Dunlap
Hi,
Hi Randy,
Quoted 1 line
A few comments for your consideration:
I merged all of you comments but two:
Quoted 1 line · Sebastian Andrzej Siewior
…
Quoted 7 lines
Quoted 7 lines
Documentation/locking/locktypes.rst new file +7
-0+rwlock_t and PREEMPT_RT +----------------------- + +On a PREEMPT_RT enabled kernel rwlock_t is mapped to a separate +implementation based on rt_mutex which changes the semantics: + + - Same changes as for spinlock_tQuoted 8 lines
Documentation/locking/locktypes.rst line 162
Quoted 4 lines
Documentation/locking/locktypes.rst new file +4
-0+ + - The implementation is not fair and can cause writer starvation under + certain circumstances. The reason for this is that a writer cannot + inherit its priority to multiple readers. Readers which are blocked^^^^^^^ I think this is backwards. Maybe more like so:
a writer cannot
bequeath or grant or bestow or pass down ... its priority to
Quoted 5 lines
So the term "inherit" is the problem. The protocol is officially called PI which is short for Priority Inheritance. Other documentation, RT-mutex for instance, is also using this term when it is referring to altering the priority of a task. For that reason I prefer to keep using this term.
OK, I get it.
Quoted 11 lines
Quoted 1 line
Quoted 1 line
+ on a writer fully support the priority inheritance protocol.
…
Quoted 8 lines
Documentation/locking/locktypes.rst line 203
Quoted 5 lines
Documentation/locking/locktypes.rst new file +5
-0+raw_spinlock_t +-------------- + +As raw_spinlock_t locking disables preemption and eventually interrupts the +code inside the critical region has to be careful to avoid calls into codeCan I buy a comma in there somewhere, please?
I don't get it as is.
Quoted 6 lines
What about
| As
raw_spinlock_tlocking disables preemption, and eventually interrupts, the | code inside the critical region has to be careful to avoid calls into codeany better?
Yes.
thanks.
Signature · 1 line
~Randy
PATCH 2/9 timekeeping: Split jiffies seqlock
In reply to Sebastian Andrzej SiewiorFrom: Thomas Gleixner <[email protected]>
seqlock consists of a sequence counter and a spinlock_t which is used to serialize the writers. spinlock_t is substituted by a "sleeping" spinlock on PREEMPT_RT enabled kernels which breaks the usage in the timekeeping code as the writers are executed in hard interrupt and therefore non-preemptible context even on PREEMPT_RT.
The spinlock in seqlock cannot be unconditionally replaced by a raw_spinlock_t as many seqlock users have nesting spinlock sections or other code which is not suitable to run in truly atomic context on RT.
Instead of providing a raw_seqlock API for a single use case, open code the seqlock for the jiffies use case and implement it with a raw_spinlock_t and a sequence counter.
--- kernel/time/jiffies.c | 7 ++++--- kernel/time/tick-common.c | 10 ++++++---- kernel/time/tick-sched.c | 19 ++++++++++++------- kernel/time/timekeeping.c | 6 ++++-- kernel/time/timekeeping.h | 3 ++- 5 files changed, 28 insertions(+), 17 deletions(-)
kernel/time/jiffies.c +4 -3
@@ -58,7 +58,8 @@ static struct clocksource clocksource_jiffies = { .max_cycles = 10, }; -__cacheline_aligned_in_smp DEFINE_SEQLOCK (jiffies_lock ); +__cacheline_aligned_in_smp DEFINE_RAW_SPINLOCK (jiffies_lock ); +__cacheline_aligned_in_smp seqcount_t jiffies_seq;#if (BITS_PER_LONG <64 )u64 get_jiffies_64 (void )
@@ -67,9 +68,9 @@ u64 get_jiffies_64(void)u64 ret;do { - seq =read_seqbegin (&jiffies_lock); + seq =read_seqcount_begin (&jiffies_seq); ret = jiffies_64; - }while (read_seqretry (&jiffies_lock, seq)); + }while (read_seqcount_retry (&jiffies_seq, seq));return ret; }EXPORT_SYMBOL (get_jiffies_64);
kernel/time/tick-common.c +6 -4
@@ -84,13 +84,15 @@ int tick_is_oneshot_available(void)static void tick_periodic (int cpu) {if (tick_do_timer_cpu == cpu) { -write_seqlock (&jiffies_lock); +raw_spin_lock (&jiffies_lock); +write_seqcount_begin (&jiffies_seq);/* Keep track of the next tick event */ tick_next_period =ktime_add (tick_next_period, tick_period);do_timer (1 ); -write_sequnlock (&jiffies_lock); +write_seqcount_end (&jiffies_seq); +raw_spin_unlock (&jiffies_lock);update_wall_time (); }
@@ -162,9 +164,9 @@ void tick_setup_periodic(struct clock_event_device *dev, int broadcast)ktime_t next;do { - seq =read_seqbegin (&jiffies_lock); + seq =read_seqcount_begin (&jiffies_seq); next = tick_next_period; - }while (read_seqretry (&jiffies_lock, seq)); + }while (read_seqcount_retry (&jiffies_seq, seq));clockevents_switch_state (dev,CLOCK_EVT_STATE_ONESHOT );
kernel/time/tick-sched.c +12 -7
@@ -65,7 +65,8 @@ static void tick_do_update_jiffies64(ktime_t now)return ;/* Reevaluate with jiffies_lock held */ -write_seqlock (&jiffies_lock); +raw_spin_lock (&jiffies_lock); +write_seqcount_begin (&jiffies_seq); delta =ktime_sub (now, last_jiffies_update);if (delta >= tick_period) {
@@ -91,10 +92,12 @@ static void tick_do_update_jiffies64(ktime_t now)/* Keep the tick_next_period variable up to date */ tick_next_period =ktime_add (last_jiffies_update, tick_period); }else { -write_sequnlock (&jiffies_lock); +write_seqcount_end (&jiffies_seq); +raw_spin_unlock (&jiffies_lock);return ; } -write_sequnlock (&jiffies_lock); +write_seqcount_end (&jiffies_seq); +raw_spin_unlock (&jiffies_lock);update_wall_time (); }
@@ -105,12 +108,14 @@ static ktime_t tick_init_jiffy_update(void) {ktime_t period; -write_seqlock (&jiffies_lock); +raw_spin_lock (&jiffies_lock); +write_seqcount_begin (&jiffies_seq);/* Did we start the jiffies update yet ? */ if (last_jiffies_update ==0 ) last_jiffies_update = tick_next_period; period = last_jiffies_update; -write_sequnlock (&jiffies_lock); +write_seqcount_end (&jiffies_seq); +raw_spin_unlock (&jiffies_lock);return period; }
@@ -676,10 +681,10 @@ static ktime_t tick_nohz_next_event(struct tick_sched *ts, int cpu)/* Read jiffies and the time when jiffies were updated last */ do { - seq =read_seqbegin (&jiffies_lock); + seq =read_seqcount_begin (&jiffies_seq); basemono = last_jiffies_update; basejiff = jiffies; - }while (read_seqretry (&jiffies_lock, seq)); + }while (read_seqcount_retry (&jiffies_seq, seq)); ts->last_jiffies = basejiff; ts->timer_expires_base = basemono;
kernel/time/timekeeping.c +4 -2
@@ -2397,8 +2397,10 @@ EXPORT_SYMBOL(hardpps);*/ void xtime_update (unsigned long ticks) { -write_seqlock (&jiffies_lock); +raw_spin_lock (&jiffies_lock); +write_seqcount_begin (&jiffies_seq);do_timer (ticks); -write_sequnlock (&jiffies_lock); +write_seqcount_end (&jiffies_seq); +raw_spin_unlock (&jiffies_lock);update_wall_time (); }
kernel/time/timekeeping.h +2 -1
@@ -25,7 +25,8 @@ static inline void sched_clock_resume(void) { }extern void do_timer (unsigned long ticks);extern void update_wall_time (void ); -extern seqlock_t jiffies_lock; +extern raw_spinlock_t jiffies_lock; +extern seqcount_t jiffies_seq;#define CS_NAME_LEN 32
Signature · 1 line
2.25.1
PATCH 3/9 pci/switchtec: Don't abuse completion wait queue for poll
In reply to Sebastian Andrzej SiewiorThe poll callback is abusing the completion wait queue and sticks it into poll_wait() to wake up pollers after a command has completed.
First of all it's a layering violation as it imposes restrictions on the inner workings of completions. Just because C allows to do so does not justify that in any way. The proper way to do such things is to post patches which extend the core infrastructure and not by silently abusing it.
Aside of that the implementation is seriously broken:
1) It cannot work with EPOLLEXCLUSIVE
2) It's racy:
poll() write()
switchtec_dev_poll() switchtec_dev_write()
poll_wait(&s->comp.wait); mrpc_queue_cmd()
init_completion(&s->comp)
init_waitqueue_head(&s->comp.wait)Replace it with a regular wait queue which removes the completion abuse and cures #1 and #2 above.
--- drivers/pci/switch/switchtec.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
drivers/pci/switch/switchtec.c +13 -9
@@ -52,10 +52,11 @@ struct switchtec_user {enum mrpc_state state; -struct completion comp; +wait_queue_head_t cmd_comp;struct kref kref;struct list_head list; +bool cmd_done;u32 cmd;u32 status;u32 return_code;
@@ -77,7 +78,7 @@ static struct switchtec_user *stuser_create(struct switchtec_dev *stdev) stuser->stdev = stdev;kref_init (&stuser->kref);INIT_LIST_HEAD (&stuser->list); -init_completion (&stuser->comp); +init_waitqueue_head (&stuser->cmd_comp); stuser->event_cnt =atomic_read (&stdev->event_cnt);dev_dbg (&stdev->dev,"%s: %p\n" , __func__, stuser);
@@ -175,7 +176,7 @@ static int mrpc_queue_cmd(struct switchtec_user *stuser)kref_get (&stuser->kref); stuser->read_len =sizeof (stuser->data);stuser_set_state (stuser,MRPC_QUEUED ); -init_completion (&stuser->comp); + stuser->cmd_done = false;list_add_tail (&stuser->list, &stdev->mrpc_queue);mrpc_cmd_submit (stdev);
@@ -222,7 +223,8 @@ static void mrpc_complete_cmd(struct switchtec_dev *stdev)memcpy_fromio (stuser->data, &stdev->mmio_mrpc->output_data, stuser->read_len); out: -complete_all (&stuser->comp); + stuser->cmd_done = true; +wake_up_interruptible (&stuser->cmd_comp);list_del_init (&stuser->list);stuser_put (stuser); stdev->mrpc_busy =0 ;
@@ -529,10 +531,11 @@ static ssize_t switchtec_dev_read(struct file *filp, char __user *data,mutex_unlock (&stdev->mrpc_mutex);if (filp->f_flags &O_NONBLOCK ) { -if (!try_wait_for_completion (&stuser->comp)) +if (!stuser->cmd_done)return -EAGAIN ; }else { - rc =wait_for_completion_interruptible (&stuser->comp); + rc =wait_event_interruptible (stuser->cmd_comp, + stuser->cmd_done);if (rc <0 )return rc; }
@@ -580,7 +583,7 @@ static __poll_t switchtec_dev_poll(struct file *filp, poll_table *wait)struct switchtec_dev *stdev = stuser->stdev;__poll_t ret =0 ; -poll_wait (filp, &stuser->comp.wait, wait); +poll_wait (filp, &stuser->cmd_comp, wait);poll_wait (filp, &stdev->event_wq, wait);if (lock_mutex_and_test_alive (stdev))
@@ -588,7 +591,7 @@ static __poll_t switchtec_dev_poll(struct file *filp, poll_table *wait)mutex_unlock (&stdev->mrpc_mutex); -if (try_wait_for_completion (&stuser->comp)) +if (stuser->cmd_done) ret |=EPOLLIN |EPOLLRDNORM ;if (stuser->event_cnt !=atomic_read (&stdev->event_cnt))
@@ -1272,7 +1275,8 @@ static void stdev_kill(struct switchtec_dev *stdev)/* Wake up and kill any users waiting on an MRPC request */ list_for_each_entry_safe (stuser, tmpuser, &stdev->mrpc_queue, list) { -complete_all (&stuser->comp); + stuser->cmd_done = true; +wake_up_interruptible (&stuser->cmd_comp);list_del_init (&stuser->list);stuser_put (stuser); }
Signature · 1 line
2.25.1
Logan Gunthorpe
maintainerRe: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 8 lines · Sebastian Andrzej Siewior
The poll callback is abusing the completion wait queue and sticks it into
poll_wait()to wake up pollers after a command has completed.First of all it's a layering violation as it imposes restrictions on the inner workings of completions. Just because C allows to do so does not justify that in any way. The proper way to do such things is to post patches which extend the core infrastructure and not by silently abusing it.
As I've said previously, I disagree with this approach. Open coding standard primitives sweeps issues under the rug and is a step backwards for code quality. Calling it a layering violation is just one opinion and if it is, the better solution would be to create an interface you find appropriate so that it isn't one.
Quoted 3 lines
Aside of that the implementation is seriously broken:
1) It cannot work with EPOLLEXCLUSIVE
Why? You don't explain this. And I don't see how this patch would change anything to do with the call to poll_wait(). All you've done is open-code the completion.
Not that it matters that much, having multiple waiters poll on this interface can pretty much never happen. It only makes sense for the process who submitted the write to poll on the interface.
Quoted 7 lines
2) It's racy:
poll()write()
switchtec_dev_poll()switchtec_dev_write()poll_wait(&s->comp.wait); mrpc_queue_cmd() init_completion(&s->comp) init_waitqueue_head(&s->comp.wait)
That's a nice catch! But wouldn't an easier solution be to change the code to use reinit_completion() instead of using the bug to justify a different change?
Thanks,
Logan
Re: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 11 lines · Logan Gunthorpe
Quoted 8 lines · Sebastian Andrzej Siewior
The poll callback is abusing the completion wait queue and sticks it into
poll_wait()to wake up pollers after a command has completed.First of all it's a layering violation as it imposes restrictions on the inner workings of completions. Just because C allows to do so does not justify that in any way. The proper way to do such things is to post patches which extend the core infrastructure and not by silently abusing it.
As I've said previously, I disagree with this approach.
Feel free to do s.
Quoted 4 lines
Open coding standard primitives sweeps issues under the rug and is a step backwards for code quality. Calling it a layering violation is just one opinion and if it is, the better solution would be to create an interface you find appropriate so that it isn't one.
There is no standard primitive which allows to poll on a completion.
You decided that this is smart to do and just because C does not allow to hide implementation details this is not a justification for this at all.
Due to the limitations of C, the kernel has to rely on the assumption that developers know and respect the difference between API and implementation.
Relying on implementation details of an interface and then arguing that this is a standard primitive for the chosen purpose is just backwards.
What's even more hillarious is that you now request that we give you a replacement interface for something which was not an interface to use in the first place.
Quoted 9 lines
Quoted 1 line
1) It cannot work with EPOLLEXCLUSIVE
Why? You don't explain this. And I don't see how this patch would change anything to do with the call to
poll_wait(). All you've done is open-code the completion.Not that it matters that much, having multiple waiters poll on this interface can pretty much never happen. It only makes sense for the process who submitted the write to poll on the interface.
It does not matter whether your envisioned usage implies that it cannot happen. Fact is that there is no restriction. That means using this with the well documented semantics of poll(2) will result in failure.
Quoted 11 lines
Quoted 7 lines
2) It's racy:
poll()write()
switchtec_dev_poll()switchtec_dev_write()poll_wait(&s->comp.wait); mrpc_queue_cmd() init_completion(&s->comp) init_waitqueue_head(&s->comp.wait)That's a nice catch! But wouldn't an easier solution be to change the code to use
reinit_completion()instead of using the bug to justify a different change?
Sure taht can be cured by changing it to reinit, but that does not cure the abuse of implementation details. As Peter, who maintains the completion code says:
Relying on implementation details of locking primitives like that is yuck.
Thanks,
tglx
Logan Gunthorpe
maintainerRe: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 30 lines · Thomas Gleixner
Quoted 11 lines · Logan Gunthorpe
Quoted 8 lines · Sebastian Andrzej Siewior
The poll callback is abusing the completion wait queue and sticks it into
poll_wait()to wake up pollers after a command has completed.First of all it's a layering violation as it imposes restrictions on the inner workings of completions. Just because C allows to do so does not justify that in any way. The proper way to do such things is to post patches which extend the core infrastructure and not by silently abusing it.
As I've said previously, I disagree with this approach.
Feel free to do s.
Quoted 4 lines
Open coding standard primitives sweeps issues under the rug and is a step backwards for code quality. Calling it a layering violation is just one opinion and if it is, the better solution would be to create an interface you find appropriate so that it isn't one.
There is no standard primitive which allows to poll on a completion.
You decided that this is smart to do and just because C does not allow to hide implementation details this is not a justification for this at all.
Due to the limitations of C, the kernel has to rely on the assumption that developers know and respect the difference between API and implementation.
Quoted 6 lines
Relying on implementation details of an interface and then arguing that this is a standard primitive for the chosen purpose is just backwards.
What's even more hillarious is that you now request that we give you a replacement interface for something which was not an interface to use in the first place.
I'm in awe at the lack of professionalism in your emails. If you bothered to edit out the ad hominems, you might have noticed that nobody has yet described how the poll interface fails here (with EPOLLEXCLUSIVE) or how replacing one wait queue for another fixes the purported problem.
We clearly disagree on what's considered appropriate usage of the completion helper (calling it a "locking primitive" is a bit disingenuous) and it doesn't sound like that's going to change. I hold no power here, but you aren't going to bully me into giving this patch an Ack or into silencing my opinion on the matter.
I'd prefer it if you submit a patch that's honest about what it's trying to accomplish and why (ie. it doesn't masquerade as being necessary to fix a bug). I also ask that you accept that I'm within my right to voice my dissent. If, after that, Bjorn chooses to take your patch, then I will respect his decision. I trust that he's able to read behind the personal attacks and look only at the technical issues.
Logan
Re: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 6 lines · Logan Gunthorpe
On 2020-03-13 6:23 p.m., Thomas Gleixner wrote:
I'm in awe at the lack of professionalism in your emails. If you bothered to edit out the ad hominems, you might have noticed that nobody has yet described how the poll interface fails here (with EPOLLEXCLUSIVE) or how replacing one wait queue for another fixes the purported problem.
I merily stated an opinion, but if you consider this an ad hominem attack, then let me ensure you this wasn't my intention and accept my apology.
Thanks,
tglx
Logan Gunthorpe
maintainerRe: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 11 lines · Thomas Gleixner
Quoted 6 lines · Logan Gunthorpe
On 2020-03-13 6:23 p.m., Thomas Gleixner wrote:
I'm in awe at the lack of professionalism in your emails. If you bothered to edit out the ad hominems, you might have noticed that nobody has yet described how the poll interface fails here (with EPOLLEXCLUSIVE) or how replacing one wait queue for another fixes the purported problem.I merily stated an opinion, but if you consider this an ad hominem attack, then let me ensure you this wasn't my intention and accept my apology.
A technical opinion, and a valid argument, does *not* involve telling me what my decision process was ("You decided this was smart to do"), or mocking it as "hillarious" (sic).
Your actual opinion was drowned out by these attacks. And, while valid, your opinion is very much subjective and I, personally, disagree with it.
I accept your apology and hope this doesn't happen again.
Thanks,
Logan
Re: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
In reply to Logan GunthorpeQuoted 4 lines · Logan Gunthorpe
Quoted 1 line · Sebastian Andrzej Siewior
1) It cannot work with EPOLLEXCLUSIVE
Why? You don't explain this.
man epoll_ctt(2)
EPOLLEXCLUSIVE (since Linux 4.5)
Sets an exclusive wakeup mode for the epoll file descriptor that is being attached to the target file descriptor, fd. When a wakeup event occurs and multiple epoll file descriptors are attached to the same target file using EPOLLEXCLUSIVE, one or more of the epoll file descriptors will receive an event with epoll_wait(2).
As this uses complete_all() there is no distinction possible, because complete_all() wakes up everything.
Quoted 2 lines
And I don't see how this patch would change anything to do with the call to
poll_wait(). All you've done is open-code the completion.
wake_up_interruptible(x) resolves to:
__wake_up(x, TASK_INTERRUPTIBLE, 1, NULL)
which wakes exactly 1 exclusive waiter.
Also the other way round is just working because the waker side uses complete_all(). Why? Because completion internally defaults to exclusive mode and complete() wakes exactly one exlusive waiter.
There is a conceptual difference and while it works for that particular purpose to some extent it's not suitable as a general wait notification construct.
Thanks,
tglx
Logan Gunthorpe
maintainerRe: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 28 lines · Thomas Gleixner
Quoted 4 lines · Logan Gunthorpe
Quoted 1 line · Sebastian Andrzej Siewior
1) It cannot work with EPOLLEXCLUSIVE
Why? You don't explain this.
man
epoll_ctt(2)EPOLLEXCLUSIVE (since Linux 4.5)
Sets an exclusive wakeup mode for the epoll file descriptor that is being attached to the target file descriptor, fd. When a wakeup event occurs and multiple epoll file descriptors are attached to the same target file using EPOLLEXCLUSIVE, one or more of the epoll file descriptors will receive an event with
epoll_wait(2).As this uses
complete_all()there is no distinction possible, becausecomplete_all()wakes up everything.Quoted 2 lines
And I don't see how this patch would change anything to do with the call to
poll_wait(). All you've done is open-code the completion.
wake_up_interruptible(x)resolves to:
__wake_up(x, TASK_INTERRUPTIBLE, 1, NULL)which wakes exactly 1 exclusive waiter.
Quoted 7 lines
Also the other way round is just working because the waker side uses
complete_all(). Why? Because completion internally defaults to exclusive mode andcomplete()wakes exactly one exlusive waiter.There is a conceptual difference and while it works for that particular purpose to some extent it's not suitable as a general wait notification construct.
Ok, I now understand this point. That's exceedingly subtle.
I certainly would not agree that this qualifies as "seriously broken", and I'm not even sure I'd agree that this actually violates the semantics of poll() seeing the man page clearly states that with EPOLLEXCLUSIVE set, "one or more" pollers will be woken up. So waking up all of them is still allowed. Ensuring fewer pollers wake up is just an optimization to avoid the thundering herd problem which users of this interface are very unlikely to ever have (I can confidently tell you that none have this problem now).
If we do want to say that all poll_wait() users *must* respect EPOLLEXCLUSIVE, we should at least have some documentation saying that combining poll_wait() with wake_up_all() (or similar) is not allowed. A *very* quick check finds there's at least a few drivers doing this:
drivers/char/ipmi/ipmb_dev_int.c
drivers/dma-buf/sync_file.c
drivers/gpu/vga/vgaarb.c
(That's just looking at the drivers tree, up to "G".)
Finally, since we seem to back to more reasonable discussion, I will make this point: it's fairly common for wait queue users to directly use the spinlock from within wait_queue_head_t without an interface (even completion.c does it). How are developers supposed to know when an interface is required and when it's not? Sometimes using "implementation" details interface-free is standard practice, but other times it's "yuck" and will illicit ire from other developers? Is it valid to use completion.wait.lock? Where's the line?
Logan
Re: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Logan,
Quoted 3 lines · Logan Gunthorpe
Quoted 1 line · Thomas Gleixner
Logan Gunthorpe <[email protected]> writes:
I certainly would not agree that this qualifies as "seriously broken",
I concede that this is the wrong wording, but chasing things like this and constantly mopping up code is not necessarily keeping my mood up all the time.
Quoted 7 lines
and I'm not even sure I'd agree that this actually violates the semantics of
poll()seeing the man page clearly states that with EPOLLEXCLUSIVE set, "one or more" pollers will be woken up. So waking up all of them is still allowed. Ensuring fewer pollers wake up is just an optimization to avoid the thundering herd problem which users of this interface are very unlikely to ever have (I can confidently tell you that none have this problem now).
I can see the point, but in general we should just strive for keeping the interfaces consistent and not subject to interpretation by individual developers. Maybe in your case the probability that someone wants to use it in that way is close to zero, but we've been surprised often enough by the creativity of user space developers who came up with use cases nobody ever expected.
Quoted 8 lines
If we do want to say that all
poll_wait()users *must* respect EPOLLEXCLUSIVE, we should at least have some documentation saying that combiningpoll_wait()withwake_up_all()(or similar) is not allowed. A *very* quick check finds there's at least a few drivers doing this:
drivers/char/ipmi/ipmb_dev_int.c
drivers/dma-buf/sync_file.c
drivers/gpu/vga/vgaarb.c
Right. There is surely quite some stuff in drivers which either predates these things or slipped through review.
Quoted 4 lines
Finally, since we seem to back to more reasonable discussion, I will make this point: it's fairly common for wait queue users to directly use the spinlock from within
wait_queue_head_twithout an interface (evencompletion.cdoes it).
completions and waitqueues are both core code. Core primitives build obviously on other primitives so dependencies on the inner workings are expected to some degree, especially for real and valuable optimization reasons. Solving these dependencies when one primitive changes has limited scope.
Quoted 5 lines
How are developers supposed to know when an interface is required and when it's not? Sometimes using "implementation" details interface-free is standard practice, but other times it's "yuck" and will illicit ire from other developers? Is it valid to use completion.wait.lock? Where's the line?
That's a good question, but that question simply arises due to the fact that C does not provide proper privatizing or if you want to work around that it forces you to come up with really horrible constructs.
The waitqueue lock usage has a very long history. It goes back to 2002 when the semaphore implementation was optimized. That exposed some of the waitqueue internals which got consequently used elsewhere as well. But looking at the actual usage sites, that's a pretty limited amount. Most of them are core infrastrucure. Of course there are drivers as well which use it for questionable reasons.
In general I think that silently using implementation details just to scratch an itch or "optimizing" code is pretty much bad practice.
Especially as this has a tendency to spread out in creative ways. And this happens simply because developers often copy the next thing which looks suitable and then expand on it. As this is not necessarily caught in reviews, this can spread to the point where the core infrastructure cannot be changed anymore.
That's not a made up nightmare scenario. This happened in reality and caused me to mop up 50+ interrupt chip implementations in order to be able to make an urgently needed 10 line change to the core interrupt infrastructure. Guess what, the vast majority of instances fiddling with the core internals were either voodoo programming or plain bugs. There were a few legitimate issues, but they had been better solved in the core code upfront. Even after that cleanup a driver got merged which had #include "../../../../kernel/irg/internal.h" inside just because the code which was developed out of tree before this change had be made to "work".
That's just not a workable solution with the ever expanding code base of the kernel.
I really prefer when people come along and show the problem they want to solve - I'm completely fine with the POC hack which uses internals for that purpose - so that this can be discussed and eventually integrated into core infrastructure in one way or the other or better suitable solutions can be found.
There are other aspects to this:
- Using existing interfaces allows a reviewer to do the quick check on init, run time usage and tear down instead of wrapping his head around the special case
- Verification tooling of all sorts just works
- Improvements to the core implementation including new features are just coming for free.
I hope that clarifies where I'm coming from.
This has nothing to do with you personally, you just triggered a few sensible fuses while understandably defending your admittedly smart solution.
Thanks,
tglx
Logan Gunthorpe
maintainerRe: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
Quoted 3 lines · Thomas Gleixner
That's a good question, but that question simply arises due to the fact that C does not provide proper privatizing or if you want to work around that it forces you to come up with really horrible constructs.
Well, we do have the underscore convention. Though, I concede this code could potentially predate that. Had there been a preceding underscore, I definitely would have thought twice before touching it.
Quoted 10 lines
That's not a made up nightmare scenario. This happened in reality and caused me to mop up 50+ interrupt chip implementations in order to be able to make an urgently needed 10 line change to the core interrupt infrastructure. Guess what, the vast majority of instances fiddling with the core internals were either voodoo programming or plain bugs. There were a few legitimate issues, but they had been better solved in the core code upfront. Even after that cleanup a driver got merged which had #include "
../../../../kernel/irg/internal.h" inside just because the code which was developed out of tree before this change had be made to "work".
I get where your coming from, and it sucks having to clean up so many instances in an urgent situation. But I see this kind of cleanup work as routine, a necessary thing that happens all the time. I've done it myself a couple times before in the kernel. The hard trick is to get developers to do more of it, before it becomes a problem like the one you faced.
In my experience, what makes clean up work even harder is where developers see an interface, notice it's not a perfect fit and so open code the whole thing themselves. Then you have random buggy primitives open coded all over the place that are impossible to find. And the primitives themselves never improve or grow new interfaces because nobody knows there's a bunch of instances that require the improvement. That's a much bigger mess.
Quoted 5 lines
I really prefer when people come along and show the problem they want to solve - I'm completely fine with the POC hack which uses internals for that purpose - so that this can be discussed and eventually integrated into core infrastructure in one way or the other or better suitable solutions can be found.
Yes, and this code was a prototype at one point and went through review from a number of people in the community, and nobody complained about this. I've also been in the situation where I submitted a POC and somebody pointed out a better way (though with a few swears thrown in for good measure); but in that case, I was actually changing a primitive so it got their attention more easily.
It's impossible for the maintainer of a primitive to review all the use cases of every primitive when new code gets merged. But at least if new code uses/abuses the primitive they will eventually come to light and can be cleaned up as appropriate with a holistic view.
Quoted 1 line
I hope that clarifies where I'm coming from.
Yes, I understood your point. And I concede that a completion is a pretty trivial primitive to open code and the change is not really worth any further fight. If the patch gets merged (preferably with a reworked commit message), I will not complain.
Quoted 3 lines
This has nothing to do with you personally, you just triggered a few sensible fuses while understandably defending your admittedly smart solution.
Thank you.
Logan
Re: [PATCH 3/9] pci/switchtec: Don't abuse completion wait queue for poll
In reply to Sebastian Andrzej SiewiorQuoted 21 lines · Sebastian Andrzej Siewior
The poll callback is abusing the completion wait queue and sticks it into
poll_wait()to wake up pollers after a command has completed.First of all it's a layering violation as it imposes restrictions on the inner workings of completions. Just because C allows to do so does not justify that in any way. The proper way to do such things is to post patches which extend the core infrastructure and not by silently abusing it.
Aside of that the implementation is seriously broken:
1) It cannot work with EPOLLEXCLUSIVE
2) It's racy:
poll()write()
switchtec_dev_poll()switchtec_dev_write()poll_wait(&s->comp.wait); mrpc_queue_cmd() init_completion(&s->comp) init_waitqueue_head(&s->comp.wait)
Quoted 8 lines
Replace it with a regular wait queue which removes the completion abuse and cures #1 and #2 above.
Cc Kurt Schwemmer <[email protected]>Cc Logan Gunthorpe <[email protected]>Cc Bjorn Helgaas <[email protected]>Signed-off-by Sebastian Andrzej Siewior <[email protected]>
Relying on implementation details of locking primitives like that is yuck.
PATCH 4/9 sched/swait: Prepare usage in completions
In reply to Sebastian Andrzej SiewiorFrom: Thomas Gleixner <[email protected]>
As a preparation to use simple wait queues for completions:
- Provide swake_up_all_locked() to support complete_all()
- Make __prepare_to_swait() public available
This is done to enable the usage of complete() within truly atomic contexts on a PREEMPT_RT enabled kernel.
--- kernel/sched/sched.h | 3 +++ kernel/sched/swait.c | 8 +++++++- 2 files changed, 10 insertions(+), 1 deletion(-)
kernel/sched/sched.h +3 -0
@@ -2492,3 +2492,6 @@ static inline bool is_per_cpu_kthread(struct task_struct *p) return true; }#endif + +void swake_up_all_locked (struct swait_queue_head *q); +void __prepare_to_swait (struct swait_queue_head *q,struct swait_queue *wait);
kernel/sched/swait.c +7 -1
@@ -32,6 +32,12 @@ void swake_up_locked(struct swait_queue_head *q) }EXPORT_SYMBOL (swake_up_locked); +void swake_up_all_locked (struct swait_queue_head *q) +{ +while (!list_empty (&q->task_list)) +swake_up_locked (q); +} +void swake_up_one (struct swait_queue_head *q) {unsigned long flags;
@@ -69,7 +75,7 @@ void swake_up_all(struct swait_queue_head *q) }EXPORT_SYMBOL (swake_up_all); -static void __prepare_to_swait (struct swait_queue_head *q,struct swait_queue *wait) +void __prepare_to_swait (struct swait_queue_head *q,struct swait_queue *wait) { wait->task = current;if (list_empty (&wait->task_list))
Signature · 1 line
2.25.1
Re: [PATCH 4/9] sched/swait: Prepare usage in completions
kernel/sched/swait.c void swake_up_locked(struct swait_queue_head *q) · line 39
Quoted 6 lines · Sebastian Andrzej Siewior
kernel/sched/swait.c +5
-0+void swake_up_all_locked (struct swait_queue_head *q) +{ +while (!list_empty (&q->task_list)) +swake_up_locked (q); +}
This one wants a big fat comment along with the usage site in the completions.
Yes, it's my fault that I forgot to double check this. Will send a delta patch when brain is more awake.
Thanks,
tglx
PATCH 5/9 completion: Use simple wait queues
In reply to Sebastian Andrzej SiewiorFrom: Thomas Gleixner <[email protected]>
completion uses a wait_queue_head_t to enqueue waiters.
wait_queue_head_t contains a spinlock_t to protect the list of waiters which excludes it from being used in truly atomic context on a PREEMPT_RT enabled kernel.
The spinlock in the wait queue head cannot be replaced by a raw_spinlock because:
- wait queues can have custom wakeup callbacks, which acquire other spinlock_t locks and have potentially long execution times
- wake_up() walks an unbounded number of list entries during the wake up and may wake an unbounded number of waiters.
For simplicity and performance reasons complete() should be usable on PREEMPT_RT enabled kernels.
completions do not use custom wakeup callbacks and are usually single waiter, except for a few corner cases.
Replace the wait queue in the completion with a simple wait queue (swait), which uses a raw_spinlock_t for protecting the waiter list and therefore is safe to use inside truly atomic regions on PREEMPT_RT.
complete_all() might cause unbound latencies with a large number of waiters being woken at once, but most complete_all() usage sites are either in testing or initialization code or have only a really small number of concurrent waiters which for now does not cause a latency problem. Keep it simple for now.
--- arch/powerpc/platforms/ps3/device-init.c | 4 +-- .../wireless/intersil/orinoco/orinoco_usb.c | 4 +-- drivers/usb/gadget/function/f_fs.c | 2 +- drivers/usb/gadget/legacy/inode.c | 4 +-- include/linux/completion.h | 8 ++--- kernel/sched/completion.c | 36 ++++++++++--------- 6 files changed, 30 insertions(+), 28 deletions(-)
arch/powerpc/platforms/ps3/device-init.c +2 -2
@@ -738,8 +738,8 @@ static int ps3_notification_read_write(struct ps3_notification_device *dev, }pr_debug ("%s:%u: notification %s issued\n" , __func__, __LINE__, op); - res =wait_event_interruptible (dev->done.wait, - dev->done.done ||kthread_should_stop ()); + res =swait_event_interruptible_exclusive (dev->done.wait, + dev->done.done ||kthread_should_stop ());if (kthread_should_stop ()) res = -EINTR ;if (res) {
drivers/net/wireless/intersil/orinoco/orinoco_usb.c +2 -2
@@ -693,8 +693,8 @@ static void ezusb_req_ctx_wait(struct ezusb_priv *upriv,while (!ctx->done.done && msecs--)udelay (1000 ); }else { -wait_event_interruptible (ctx->done.wait, - ctx->done.done); +swait_event_interruptible_exclusive (ctx->done.wait, + ctx->done.done); }break ;default :
drivers/usb/gadget/function/f_fs.c +1 -1
@@ -1703,7 +1703,7 @@ static void ffs_data_put(struct ffs_data *ffs)pr_info ("%s(): freeing\n" , __func__);ffs_data_clear (ffs);BUG_ON (waitqueue_active (&ffs->ev.waitq) || -waitqueue_active (&ffs->ep0req_completion.wait) || +swait_active (&ffs->ep0req_completion.wait) ||waitqueue_active (&ffs->wait));destroy_workqueue (ffs->io_completion_wq);kfree (ffs->dev_name);
drivers/usb/gadget/legacy/inode.c +2 -2
@@ -344,7 +344,7 @@ ep_io (struct ep_data *epdata, void *buf, unsigned len)spin_unlock_irq (&epdata->dev->lock);if (likely (value ==0 )) { - value =wait_event_interruptible (done.wait, done.done); + value =swait_event_interruptible_exclusive (done.wait, done.done);if (value !=0 ) {spin_lock_irq (&epdata->dev->lock);if (likely (epdata->ep !=NULL )) {
@@ -353,7 +353,7 @@ ep_io (struct ep_data *epdata, void *buf, unsigned len)usb_ep_dequeue (epdata->ep, epdata->req);spin_unlock_irq (&epdata->dev->lock); -wait_event (done.wait, done.done); +swait_event_exclusive (done.wait, done.done);if (epdata->status == -ECONNRESET ) epdata->status = -EINTR ; }else {
include/linux/completion.h +4 -4
@@ -9,7 +9,7 @@* See kernel/sched/completion.c for details. */ -#include <linux/wait.h> +#include <linux/swait.h> /* * struct completion - structure used to maintain state for a "completion"
@@ -25,7 +25,7 @@*/ struct completion {unsigned int done; -wait_queue_head_t wait; +struct swait_queue_head wait; };#define init_completion_map (x, m) __init_completion(x)
@@ -34,7 +34,7 @@ static inline void complete_acquire(struct completion *x) {}static inline void complete_release (struct completion *x) {}#define COMPLETION_INITIALIZER (work) \ - { 0, __WAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) }#define COMPLETION_INITIALIZER_ONSTACK_MAP (work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); }))
@@ -85,7 +85,7 @@ static inline void complete_release(struct completion *x) {}static inline void __init_completion (struct completion *x) { x->done =0 ; -init_waitqueue_head (&x->wait); +init_swait_queue_head (&x->wait); }/**
kernel/sched/completion.c +19 -17
@@ -29,12 +29,12 @@ void complete(struct completion *x) {unsigned long flags; -spin_lock_irqsave (&x->wait.lock, flags); +raw_spin_lock_irqsave (&x->wait.lock, flags);if (x->done !=UINT_MAX ) x->done++; -__wake_up_locked (&x->wait,TASK_NORMAL ,1 ); -spin_unlock_irqrestore (&x->wait.lock, flags); +swake_up_locked (&x->wait); +raw_spin_unlock_irqrestore (&x->wait.lock, flags); }EXPORT_SYMBOL (complete);
@@ -58,10 +58,12 @@ void complete_all(struct completion *x) {unsigned long flags; -spin_lock_irqsave (&x->wait.lock, flags); +WARN_ON (irqs_disabled ()); + +raw_spin_lock_irqsave (&x->wait.lock, flags); x->done =UINT_MAX ; -__wake_up_locked (&x->wait,TASK_NORMAL ,0 ); -spin_unlock_irqrestore (&x->wait.lock, flags); +swake_up_all_locked (&x->wait); +raw_spin_unlock_irqrestore (&x->wait.lock, flags); }EXPORT_SYMBOL (complete_all);
@@ -70,20 +72,20 @@ do_wait_for_common(struct completion *x, long (*action)(long), long timeout, int state) { if (!x->done) { -DECLARE_WAITQUEUE (wait ,current ); +DECLARE_SWAITQUEUE (wait ); -__add_wait_queue_entry_tail_exclusive (&x->wait, &wait);do {if (signal_pending_state (state, current)) { timeout = -ERESTARTSYS ;break ; } +__prepare_to_swait (&x->wait, &wait);__set_current_state (state); -spin_unlock_irq (&x->wait.lock); +raw_spin_unlock_irq (&x->wait.lock); timeout =action (timeout); -spin_lock_irq (&x->wait.lock); +raw_spin_lock_irq (&x->wait.lock); }while (!x->done && timeout); -__remove_wait_queue (&x->wait, &wait); +__finish_swait (&x->wait, &wait);if (!x->done)return timeout; }
@@ -100,9 +102,9 @@ __wait_for_common(struct completion *x,complete_acquire (x); -spin_lock_irq (&x->wait.lock); +raw_spin_lock_irq (&x->wait.lock); timeout =do_wait_for_common (x, action, timeout, state); -spin_unlock_irq (&x->wait.lock); +raw_spin_unlock_irq (&x->wait.lock);complete_release (x);
@@ -291,12 +293,12 @@ bool try_wait_for_completion(struct completion *x)if (!READ_ONCE (x->done))return false; -spin_lock_irqsave (&x->wait.lock, flags); +raw_spin_lock_irqsave (&x->wait.lock, flags);if (!x->done) ret = false;else if (x->done !=UINT_MAX ) x->done--; -spin_unlock_irqrestore (&x->wait.lock, flags); +raw_spin_unlock_irqrestore (&x->wait.lock, flags);return ret; }EXPORT_SYMBOL (try_wait_for_completion);
@@ -322,8 +324,8 @@ bool completion_done(struct completion *x)* otherwise we can end up freeing the completion before complete() * is done referencing it. */ -spin_lock_irqsave (&x->wait.lock, flags); -spin_unlock_irqrestore (&x->wait.lock, flags); +raw_spin_lock_irqsave (&x->wait.lock, flags); +raw_spin_unlock_irqrestore (&x->wait.lock, flags);return true; }EXPORT_SYMBOL (completion_done);
Signature · 1 line
2.25.1
Re: [PATCH 5/9] completion: Use simple wait queues
Quoted 2 lines · Sebastian Andrzej Siewior
Replace the wait queue in the completion with a simple wait queue (swait),
This is almost certainly completely and utterly wrong.
Every time somebody uses those horrible swait queues, the end result is buggy.
Don't do it.
And most definitely, don't do it like this, which seems to be entirely mindlessly just randomly changing things by some brute force in multiple places.
The swait semantics are completely different from the normal wait-queue semantics, and generally not in good ways.
There's a *REASON* why the comment at the top of <linux/swait.h> starts with
* BROKEN wait-queues.
*
* These "simple" wait-queues are broken garbage, and should never be
* used. The comments below claim that they are "similar" to regular
* wait-queues, but the semantics are actually completely different, and
* every single user we have ever had has been buggy (or pointless).
and before you do a conversion, you need to spend a _lot_ of time thinking about why that is the case.
And _after_ you do the conversion, you damn well need to explain why it's safe. Not just state that it's a good idea.
For example, this patch just randomly changes wait events to the swait event _exclusive_ waits. With not a single explanation of why that would be ok.
I want an explanation for EVERY SINGLE CASE. Because people have done this kind of conversion before, and it's been buggy garbage before. I want to see that people actually thought about what the semantic differences were, and _documented_ that thinking process.
Linus
Re: [PATCH 5/9] completion: Use simple wait queues
Quoted 8 lines · Linus Torvalds
On Fri, Mar 13, 2020 at 10:47 AM Sebastian Andrzej Siewior <[email protected]> wrote:
and before you do a conversion, you need to spend a _lot_ of time thinking about why that is the case.And _after_ you do the conversion, you damn well need to explain why it's safe. Not just state that it's a good idea.
Quoted 8 lines
For example, this patch just randomly changes wait events to the swait event _exclusive_ waits. With not a single explanation of why that would be ok.
I want an explanation for EVERY SINGLE CASE. Because people have done this kind of conversion before, and it's been buggy garbage before. I want to see that people actually thought about what the semantic differences were, and _documented_ that thinking process.
My bad. I'll rework the changelog so it contains the proof that the result is semantical and functional equivalent.
Thanks,
tglx
PATCH 6/9 lockdep: Introduce wait-type checks
In reply to Sebastian Andrzej SiewiorFrom: Peter Zijlstra <[email protected]>
Extend lockdep to validate lock wait-type context.
The current wait-types are:
LD_WAIT_FREE, /* wait free, rcu etc.. */ LD_WAIT_SPIN, /* spin loops, raw_spinlock_t etc.. */ LD_WAIT_CONFIG, /* CONFIG_PREEMPT_LOCK, spinlock_t etc.. */ LD_WAIT_SLEEP, /* sleeping locks, mutex_t etc.. */
Where lockdep validates that the current lock (the one being acquired) fits in the current wait-context (as generated by the held stack).
This ensures that there is no attempt to acquire mutexes while holding spinlocks, to acquire spinlocks while holding raw_spinlocks and so on. In other words, its a more fancy might_sleep().
Obviously RCU made the entire ordeal more complex than a simple single value test because RCU can be acquired in (pretty much) any context and while it presents a context to nested locks it is not the same as it got acquired in.
Therefore its necessary to split the wait_type into two values, one representing the acquire (outer) and one representing the nested context (inner). For most 'normal' locks these two are the same.
[ To make static initialization easier we have the rule that:
.outer == INV means .outer == .inner; because INV == 0. ]
It further means that its required to find the minimal .inner of the held stack to compare against the outer of the new lock; because while 'normal' RCU presents a CONFIG type to nested locks, if it is taken while already holding a SPIN type it obviously doesn't relax the rules.
Below is an example output generated by the trivial test code:
raw_spin_lock(&foo); spin_lock(&bar); spin_unlock(&bar); raw_spin_unlock(&foo);
[ BUG: Invalid wait context ]
-----------------------------
swapper/0/1 is trying to lock:
ffffc90000013f20 (&bar){....}-{3:3}, at: kernel_init+0xdb/0x187 other info that might help us debug this:
1 lock held by swapper/0/1:
#0: ffffc90000013ee0 (&foo){+.+.}-{2:2}, at: kernel_init+0xd1/0x187
The way to read it is to look at the new -{n,m} part in the lock description; -{3:3} for the attempted lock, and try and match that up to the held locks, which in this case is the one: -{2,2}.
This tells that the acquiring lock requires a more relaxed environment than presented by the lock stack.
Currently only the normal locks and RCU are converted, the rest of the lockdep users defaults to .inner = INV which is ignored. More conversions can be done when desired.
The check for spinlock_t nesting is not enabled by default. It's a separate config option for now as there are known problems which are currently addressed. The config option allows to identify these problems and to verify that the solutions found are indeed solving them.
The config switch will be removed and the checks will permanently enabled once the vast majority of issues has been addressed.
[ bigeasy: Move LD_WAIT_FREE,… out of CONFIG_LOCKDEP to avoid compile
failure with CONFIG_DEBUG_SPINLOCK + !CONFIG_LOCKDEP]
[ tglx: Add the config option ]
--- include/linux/irqflags.h | 8 +- include/linux/lockdep.h | 71 ++++++++++++++--- include/linux/mutex.h | 7 +- include/linux/rwlock_types.h | 6 +- include/linux/rwsem.h | 6 +- include/linux/sched.h | 1 + include/linux/spinlock.h | 35 ++++++--- include/linux/spinlock_types.h | 24 +++++- kernel/irq/handle.c | 7 ++ kernel/locking/lockdep.c | 132 ++++++++++++++++++++++++++++++-- kernel/locking/mutex-debug.c | 2 +- kernel/locking/rwsem.c | 2 +- kernel/locking/spinlock_debug.c | 6 +- kernel/rcu/update.c | 24 ++++-- lib/Kconfig.debug | 17 ++++ 15 files changed, 301 insertions(+), 47 deletions(-)
include/linux/irqflags.h +7 -1
@@ -37,7 +37,12 @@# define trace_softirqs_enabled (p) ((p)->softirqs_enabled)# define trace_hardirq_enter () \ do { \ - current->hardirq_context++; \ + if (!current->hardirq_context++) \ + current->hardirq_threaded = 0; \ +} while (0) +# define trace_hardirq_threaded () \ +do { \ + current->hardirq_threaded = 1; \ } while (0)# define trace_hardirq_exit () \ do { \
@@ -59,6 +64,7 @@ do { \# define trace_hardirqs_enabled (p) 0# define trace_softirqs_enabled (p) 0# define trace_hardirq_enter () do { } while (0) +# define trace_hardirq_threaded () do { } while (0)# define trace_hardirq_exit () do { } while (0)# define lockdep_softirq_enter () do { } while (0)# define lockdep_softirq_exit () do { } while (0)
include/linux/lockdep.h +60 -11
110 lines not on this page · raw message
include/linux/mutex.h +5 -2
@@ -109,8 +109,11 @@ do { \ } while (0)#ifdef CONFIG_DEBUG_LOCK_ALLOC -# define __DEP_MAP_MUTEX_INITIALIZER (lockname) \ - , .dep_map = { .name = #lockname } +# define __DEP_MAP_MUTEX_INITIALIZER (lockname) \ + , .dep_map = { \ + .name = #lockname, \ + .wait_type_inner = LD_WAIT_SLEEP, \ + }#else # define __DEP_MAP_MUTEX_INITIALIZER (lockname)#endif
include/linux/rwlock_types.h +5 -1
@@ -22,7 +22,11 @@ typedef struct {#define RWLOCK_MAGIC 0xdeaf1eed#ifdef CONFIG_DEBUG_LOCK_ALLOC -# define RW_DEP_MAP_INIT (lockname) .dep_map = { .name = #lockname } +# define RW_DEP_MAP_INIT (lockname) \ + .dep_map = { \ + .name = #lockname, \ + .wait_type_inner = LD_WAIT_CONFIG, \ + }#else # define RW_DEP_MAP_INIT (lockname)#endif
include/linux/rwsem.h +5 -1
@@ -71,7 +71,11 @@ static inline int rwsem_is_locked(struct rw_semaphore *sem)/* Common initializer macros and functions */ #ifdef CONFIG_DEBUG_LOCK_ALLOC -# define __RWSEM_DEP_MAP_INIT (lockname) , .dep_map = { .name = #lockname } +# define __RWSEM_DEP_MAP_INIT (lockname) \ + , .dep_map = { \ + .name = #lockname, \ + .wait_type_inner = LD_WAIT_SLEEP, \ + }#else # define __RWSEM_DEP_MAP_INIT (lockname)#endif
include/linux/sched.h +1 -0
@@ -970,6 +970,7 @@ struct task_struct {#ifdef CONFIG_TRACE_IRQFLAGS unsigned int irq_events; +unsigned int hardirq_threaded;unsigned long hardirq_enable_ip;unsigned long hardirq_disable_ip;unsigned int hardirq_enable_event;
include/linux/spinlock.h +25 -10
@@ -93,12 +93,13 @@#ifdef CONFIG_DEBUG_SPINLOCK extern void __raw_spin_lock_init (raw_spinlock_t *lock,const char *name, -struct lock_class_key *key); -# define raw_spin_lock_init (lock) \ -do { \ - static struct lock_class_key __key; \ - \ - __raw_spin_lock_init((lock), #lock, &__key); \ +struct lock_class_key *key,short inner); + +# define raw_spin_lock_init (lock) \ +do { \ + static struct lock_class_key __key; \ + \ + __raw_spin_lock_init((lock), #lock, &__key, LD_WAIT_SPIN); \ } while (0)#else
@@ -327,12 +328,26 @@ static __always_inline raw_spinlock_t *spinlock_check(spinlock_t *lock)return &lock->rlock; } -#define spin_lock_init (_lock) \ -do { \ - spinlock_check(_lock); \ - raw_spin_lock_init(&(_lock)->rlock); \ +#ifdef CONFIG_DEBUG_SPINLOCK + +# define spin_lock_init (lock) \ +do { \ + static struct lock_class_key __key; \ + \ + __raw_spin_lock_init(spinlock_check(lock), \ + #lock, &__key, LD_WAIT_CONFIG); \ } while (0) +#else + +# define spin_lock_init (_lock) \ +do { \ + spinlock_check(_lock); \ + *(_lock) = __SPIN_LOCK_UNLOCKED(_lock); \ +} while (0) + +#endif + static __always_inline void spin_lock(spinlock_t *lock) {raw_spin_lock (&lock->rlock);
include/linux/spinlock_types.h +20 -4
@@ -33,8 +33,18 @@ typedef struct raw_spinlock {#define SPINLOCK_OWNER_INIT ((void *)-1L)#ifdef CONFIG_DEBUG_LOCK_ALLOC -# define SPIN_DEP_MAP_INIT (lockname) .dep_map = { .name = #lockname } +# define RAW_SPIN_DEP_MAP_INIT (lockname) \ + .dep_map = { \ + .name = #lockname, \ + .wait_type_inner = LD_WAIT_SPIN, \ + } +# define SPIN_DEP_MAP_INIT (lockname) \ + .dep_map = { \ + .name = #lockname, \ + .wait_type_inner = LD_WAIT_CONFIG, \ + }#else +# define RAW_SPIN_DEP_MAP_INIT (lockname)# define SPIN_DEP_MAP_INIT (lockname)#endif
@@ -51,7 +61,7 @@ typedef struct raw_spinlock { { \ .raw_lock = __ARCH_SPIN_LOCK_UNLOCKED, \ SPIN_DEBUG_INIT(lockname) \ - SPIN_DEP_MAP_INIT(lockname) } + RAW_SPIN_DEP_MAP_INIT(lockname) }#define __RAW_SPIN_LOCK_UNLOCKED (lockname) \ (raw_spinlock_t) __RAW_SPIN_LOCK_INITIALIZER(lockname)
@@ -72,11 +82,17 @@ typedef struct spinlock { }; } spinlock_t; +#define ___SPIN_LOCK_INITIALIZER (lockname) \ + { \ + .raw_lock = __ARCH_SPIN_LOCK_UNLOCKED, \ + SPIN_DEBUG_INIT(lockname) \ + SPIN_DEP_MAP_INIT(lockname) } +#define __SPIN_LOCK_INITIALIZER (lockname) \ - { { .rlock = __RAW_SPIN_LOCK_INITIALIZER(lockname) } } + { { .rlock = ___SPIN_LOCK_INITIALIZER(lockname) } }#define __SPIN_LOCK_UNLOCKED (lockname) \ - (spinlock_t ) __SPIN_LOCK_INITIALIZER(lockname) + (spinlock_t) __SPIN_LOCK_INITIALIZER(lockname)#define DEFINE_SPINLOCK (x) spinlock_t x = __SPIN_LOCK_UNLOCKED(x)
kernel/irq/handle.c +7 -0
@@ -145,6 +145,13 @@ irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc, unsigned int *flagsfor_each_action_of_desc (desc, action) {irqreturn_t res; +/* +* If this IRQ would be threaded under force_irqthreads, mark it so. +*/ +if (irq_settings_can_thread (desc) && + !(action->flags & (IRQF_NO_THREAD |IRQF_PERCPU |IRQF_ONESHOT ))) +trace_hardirq_threaded (); +trace_irq_handler_entry (irq, action); res = action->handler (irq, action->dev_id);trace_irq_handler_exit (irq, action, res);
kernel/locking/lockdep.c +126 -6
186 lines not on this page · raw message
kernel/locking/mutex-debug.c +1 -1
@@ -85,7 +85,7 @@ void debug_mutex_init(struct mutex *lock, const char *name,* Make sure we are not reinitializing a held lock: */ debug_check_no_locks_freed ((void *)lock,sizeof (*lock)); -lockdep_init_map (&lock->dep_map, name, key,0 ); +lockdep_init_map_wait (&lock->dep_map, name, key,0 ,LD_WAIT_SLEEP );#endif lock->magic = lock; }
kernel/locking/rwsem.c +1 -1
@@ -329,7 +329,7 @@ void __init_rwsem(struct rw_semaphore *sem, const char *name,* Make sure we are not reinitializing a held semaphore: */ debug_check_no_locks_freed ((void *)sem,sizeof (*sem)); -lockdep_init_map (&sem->dep_map, name, key,0 ); +lockdep_init_map_wait (&sem->dep_map, name, key,0 ,LD_WAIT_SLEEP );#endif #ifdef CONFIG_DEBUG_RWSEMS sem->magic = sem;
kernel/locking/spinlock_debug.c +3 -3
@@ -14,14 +14,14 @@#include <linux/export.h> void __raw_spin_lock_init (raw_spinlock_t *lock,const char *name, -struct lock_class_key *key) +struct lock_class_key *key,short inner) {#ifdef CONFIG_DEBUG_LOCK_ALLOC /* * Make sure we are not reinitializing a held lock: */ debug_check_no_locks_freed ((void *)lock,sizeof (*lock)); -lockdep_init_map (&lock->dep_map, name, key,0 ); +lockdep_init_map_wait (&lock->dep_map, name, key,0 , inner);#endif lock->raw_lock = (arch_spinlock_t )__ARCH_SPIN_LOCK_UNLOCKED; lock->magic =SPINLOCK_MAGIC ;
@@ -39,7 +39,7 @@ void __rwlock_init(rwlock_t *lock, const char *name,* Make sure we are not reinitializing a held lock: */ debug_check_no_locks_freed ((void *)lock,sizeof (*lock)); -lockdep_init_map (&lock->dep_map, name, key,0 ); +lockdep_init_map_wait (&lock->dep_map, name, key,0 ,LD_WAIT_CONFIG );#endif lock->raw_lock = (arch_rwlock_t ) __ARCH_RW_LOCK_UNLOCKED; lock->magic =RWLOCK_MAGIC ;
kernel/rcu/update.c +18 -6
@@ -227,18 +227,30 @@ core_initcall(rcu_set_runtime_mode);#ifdef CONFIG_DEBUG_LOCK_ALLOC static struct lock_class_key rcu_lock_key; -struct lockdep_map rcu_lock_map = -STATIC_LOCKDEP_MAP_INIT ("rcu_read_lock" , &rcu_lock_key); +struct lockdep_map rcu_lock_map = { + .name ="rcu_read_lock" , + .key = &rcu_lock_key, + .wait_type_outer =LD_WAIT_FREE , + .wait_type_inner =LD_WAIT_CONFIG ,/* XXX PREEMPT_RCU ? */ +};EXPORT_SYMBOL_GPL (rcu_lock_map);static struct lock_class_key rcu_bh_lock_key; -struct lockdep_map rcu_bh_lock_map = -STATIC_LOCKDEP_MAP_INIT ("rcu_read_lock_bh" , &rcu_bh_lock_key); +struct lockdep_map rcu_bh_lock_map = { + .name ="rcu_read_lock_bh" , + .key = &rcu_bh_lock_key, + .wait_type_outer =LD_WAIT_FREE , + .wait_type_inner =LD_WAIT_CONFIG ,/* PREEMPT_LOCK also makes BH preemptible */ +};EXPORT_SYMBOL_GPL (rcu_bh_lock_map);static struct lock_class_key rcu_sched_lock_key; -struct lockdep_map rcu_sched_lock_map = -STATIC_LOCKDEP_MAP_INIT ("rcu_read_lock_sched" , &rcu_sched_lock_key); +struct lockdep_map rcu_sched_lock_map = { + .name ="rcu_read_lock_sched" , + .key = &rcu_sched_lock_key, + .wait_type_outer =LD_WAIT_FREE , + .wait_type_inner =LD_WAIT_SPIN , +};EXPORT_SYMBOL_GPL (rcu_sched_lock_map);static struct lock_class_key rcu_callback_key;
lib/Kconfig.debug +17 -0
@@ -1086,6 +1086,23 @@ config PROVE_LOCKING For more details, see Documentation/locking/lockdep-design.rst. +config PROVE_RAW_LOCK_NESTING + bool "Enable raw_spinlock - spinlock nesting checks" + depends on PROVE_LOCKING + default n + help + Enable the raw_spinlock vs. spinlock nesting checks which ensure + that the lock nesting rules for PREEMPT_RT enabled kernels are + not violated. + + NOTE: There are known nesting problems. So if you enable this + option expect lockdep splats until these problems have been fully + addressed which is work in progress. This config switch allows to + identify and analyze these problems. It will be removed and the + check permanentely enabled once the main issues have been fixed. + + If unsure, select N. + config LOCK_STAT bool "Lock usage statistics" depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT
Signature · 1 line
2.25.1
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Hi all,
Quoted 69 lines · Sebastian Andrzej Siewior
From: Peter Zijlstra <[email protected]>
Extend lockdep to validate lock wait-type context.
The current wait-types are:
LD_WAIT_FREE, /* wait free, rcu etc.. */ LD_WAIT_SPIN, /* spin loops, raw_spinlock_t etc.. */ LD_WAIT_CONFIG, /* CONFIG_PREEMPT_LOCK, spinlock_t etc.. */ LD_WAIT_SLEEP, /* sleeping locks, mutex_t etc.. */Where lockdep validates that the current lock (the one being acquired) fits in the current wait-context (as generated by the held stack).
This ensures that there is no attempt to acquire mutexes while holding spinlocks, to acquire spinlocks while holding
raw_spinlocksand so on. In other words, its a more fancymight_sleep().Obviously RCU made the entire ordeal more complex than a simple single value test because RCU can be acquired in (pretty much) any context and while it presents a context to nested locks it is not the same as it got acquired in.
Therefore its necessary to split the
wait_typeinto two values, one representing the acquire (outer) and one representing the nested context (inner). For most 'normal' locks these two are the same.[ To make static initialization easier we have the rule that:
.outer == INV means .outer == .inner; because INV == 0. ]It further means that its required to find the minimal .inner of the held stack to compare against the outer of the new lock; because while 'normal' RCU presents a CONFIG type to nested locks, if it is taken while already holding a SPIN type it obviously doesn't relax the rules.
Below is an example output generated by the trivial test code:
raw_spin_lock(&foo); spin_lock(&bar); spin_unlock(&bar); raw_spin_unlock(&foo);[ BUG: Invalid wait context ]
-----------------------------
swapper/0/1 is trying to lock:
ffffc90000013f20(&bar){....}-{3:3}, at:kernel_init+0xdb/0x187other info that might help us debug this:
1 lock held by swapper/0/1:
#0:ffffc90000013ee0(&foo){+.+.}-{2:2}, at:kernel_init+0xd1/0x187The way to read it is to look at the new -{n,m} part in the lock description; -{3:3} for the attempted lock, and try and match that up to the held locks, which in this case is the one: -{2,2}.
This tells that the acquiring lock requires a more relaxed environment than presented by the lock stack.
Currently only the normal locks and RCU are converted, the rest of the lockdep users defaults to .inner = INV which is ignored. More conversions can be done when desired.
The check for
spinlock_tnesting is not enabled by default. It's a separate config option for now as there are known problems which are currently addressed. The config option allows to identify these problems and to verify that the solutions found are indeed solving them.The config switch will be removed and the checks will permanently enabled once the vast majority of issues has been addressed.
Quoted 7 lines
[ bigeasy: Move LD_WAIT_FREE,… out of
CONFIG_LOCKDEPto avoid compile
failure withCONFIG_DEBUG_SPINLOCK+ !CONFIG_LOCKDEP]
[ tglx: Add the config option ]Requested-by Thomas Gleixner <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Sebastian Andrzej Siewior <[email protected]>
On arm64 (e.g. R-Car H3 ES2.0):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679 Not tainted
+-----------------------------
+swapper/5/0 is trying to lock:
+ffffff86ff76f398 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x134/0x430 +other info that might help us debug this:
+1 lock held by swapper/5/0:
+ #0: ffffffc01103a4a0 (rcu_read_lock){....}-{1:3}, at:
rcu_lock_acquire.constprop.59+0x0/0x38
+stack backtrace:
+CPU: 5 PID: 0 Comm: swapper/5 Not tainted 5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679
+Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT) +Call trace:
+ dump_backtrace+0x0/0x180
+ show_stack+0x14/0x1c
+ dump_stack+0xdc/0x12c
+ __lock_acquire+0x37c/0xf9c
+ lock_acquire+0x258/0x288
+ _raw_spin_lock+0x34/0x48
+ __queue_work+0x134/0x430
+ queue_work_on+0x48/0x8c
+ timers_update_nohz+0x24/0x2c
+ tick_nohz_activate.isra.15.part.16+0x5c/0x80
+ tick_setup_sched_timer+0xe0/0xf0
+ hrtimer_run_queues+0x88/0xf8
+ run_local_timers+0x20/0x58
+ update_process_times+0x24/0x50
+ tick_periodic+0xd8/0xe8
+ tick_handle_periodic+0x30/0x98
+ arch_timer_handler_phys+0x28/0x3c
+ handle_percpu_devid_irq+0x64/0x110
+ generic_handle_irq+0x20/0x34
+ __handle_domain_irq+0x94/0x98
+ gic_handle_irq+0x78/0xbc
+ el1_irq+0xf4/0x1c0
+ arch_cpu_idle+0x38/0x54
+ default_idle_call+0x2c/0x30
+ do_idle+0x13c/0x244
+ cpu_startup_entry+0x20/0x24
+ secondary_start_kernel+0x1c4/0x1d8
On arm32 (e.g SH-Mobile AG5, using Cortex-A9 TWD):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-kzm9g-09424-g2698719b4c4f35db-dirty #253 Not tainted
+-----------------------------
+swapper/0/0 is trying to lock:
+dfbc5250 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x14c/0x4d0 +other info that might help us debug this:
+1 lock held by swapper/0/0:
+ #0: c0a17074 (rcu_read_lock){....}-{1:3}, at:
rcu_lock_acquire.constprop.33+0x0/0x38
+stack backtrace:
+CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.6.0-kzm9g-09424-g2698719b4c4f35db-dirty #253
+Hardware name: Generic SH73A0 (Flattened Device Tree) +[<c010f948>] (unwind_backtrace) from [<c010bbf0>] (show_stack+0x10/0x14) +[<c010bbf0>] (show_stack) from [<c05c3e84>] (dump_stack+0xa8/0xe0) +[<c05c3e84>] (dump_stack) from [<c017036c>] (__lock_acquire+0x398/0x1568) +[<c017036c>] (__lock_acquire) from [<c0171e90>] (lock_acquire+0x274/0x2ac) +[<c0171e90>] (lock_acquire) from [<c05dfb90>] (_raw_spin_lock+0x40/0x50) +[<c05dfb90>] (_raw_spin_lock) from [<c013adc8>] (__queue_work+0x14c/0x4d0) +[<c013adc8>] (__queue_work) from [<c013b194>] (queue_work_on+0x48/0x6c) +[<c013b194>] (queue_work_on) from [<c01a89e8>] (tick_setup_sched_timer+0x148/0x188)
+[<c01a89e8>] (tick_setup_sched_timer) from [<c0197ad8>] (hrtimer_run_queues+0x74/0x114)
+[<c0197ad8>] (hrtimer_run_queues) from [<c0195e38>] (run_local_timers+0x14/0x54)
+[<c0195e38>] (run_local_timers) from [<c0195e9c>] (update_process_times+0x24/0x54)
+[<c0195e9c>] (update_process_times) from [<c01a5520>] (tick_handle_periodic+0x28/0xa0)
+[<c01a5520>] (tick_handle_periodic) from [<c010eaf8>] (twd_handler+0x2c/0x38) +[<c010eaf8>] (twd_handler) from [<c01829c0>] (handle_percpu_devid_irq+0x58/0xfc)
+[<c01829c0>] (handle_percpu_devid_irq) from [<c017ccc4>] (generic_handle_irq+0x28/0x38)
+[<c017ccc4>] (generic_handle_irq) from [<c017d2fc>] (__handle_domain_irq+0x90/0xa0)
+[<c017d2fc>] (__handle_domain_irq) from [<c0392d70>] (gic_handle_irq+0x58/0x90) +[<c0392d70>] (gic_handle_irq) from [<c0101ab0>] (__irq_svc+0x70/0x98) +Exception stack(0xc0a01f40 to 0xc0a01f88)
+1f40: 00000001 00000006 c0a0bac0 00000000 00000001 ffffe000 c0a08ea8 c0a94532 +1f60: c0a08eec 00000001 c09278e0 00000000 ffffe000 c0a01f90 c0108448 c010844c +1f80: 60000013 ffffffff
+[<c0101ab0>] (__irq_svc) from [<c010844c>] (arch_cpu_idle+0x20/0x3c) +[<c010844c>] (arch_cpu_idle) from [<c0152350>] (do_idle+0xe8/0x13c) +[<c0152350>] (do_idle) from [<c0152718>] (cpu_startup_entry+0x18/0x1c) +[<c0152718>] (cpu_startup_entry) from [<c0900d54>] (start_kernel+0x3f0/0x49c) +[<c0900d54>] (start_kernel) from [<00000000>] (0x0)
I also see it on other arm32 platforms using Renesas-specific timers, but I'll ignore those until the issues with "standard" ARM timers have been resolved ;-)
Thanks!
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Quoted 1 line · Geert Uytterhoeven
Hi all,
Hi Geert,
Review
Quoted 8 lines
On arm64 (e.g. R-Car H3 ES2.0):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679 Not tainted
+-----------------------------(quoted diff) +2
-0+swapper/5/0 is trying to lock: +ffffff86ff76f398 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x134/0x430
The pool->lock is not yet converted.
…
Review
Quoted 8 lines
On arm32 (e.g SH-Mobile AG5, using Cortex-A9 TWD):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-kzm9g-09424-g2698719b4c4f35db-dirty #253 Not tainted
+-----------------------------(quoted diff) +2
-0+swapper/0/0 is trying to lock: +dfbc5250 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x14c/0x4d0
same issue.
Quoted 3 lines
I also see it on other arm32 platforms using Renesas-specific timers, but I'll ignore those until the issues with "standard" ARM timers have been resolved ;-)
There are some known culprits, like the work infrastructure. The printk is another one. From Kconfig:
| NOTE: There are known nesting problems. So if you enable this | option expect lockdep splats until these problems have been fully | addressed which is work in progress. This config switch allows to | identify and analyze these problems. It will be removed and the | check permanentely enabled once the main issues have been fixed.
Quoted 7 lines
Thanks!
Gr{oetje,eeting}s,
Geert
Sebastian
Peter Zijlstra
maintainerRe: [PATCH 6/9] lockdep: Introduce wait-type checks
In reply to Geert UytterhoevenQuoted 19 lines · Geert Uytterhoeven
On arm64 (e.g. R-Car H3 ES2.0):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679 Not tainted
+-----------------------------(quoted diff) +5
-0+swapper/5/0 is trying to lock: +ffffff86ff76f398 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x134/0x430 +other info that might help us debug this: +1 lock held by swapper/5/0: + #0: ffffffc01103a4a0 (rcu_read_lock){....}-{1:3}, at:rcu_lock_acquire.constprop.59+0x0/0x38
(quoted diff) +2
-0+stack backtrace: +CPU: 5 PID: 0 Comm: swapper/5 Not tainted5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679
(quoted diff) +4
-0+Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT) +Call trace: + dump_backtrace+0x0/0x180 + show_stack+0x14/0x1c
Review
Quoted 10 lines
(quoted diff) +10
-0+ dump_stack+0xdc/0x12c + __lock_acquire+0x37c/0xf9c + lock_acquire+0x258/0x288 + _raw_spin_lock+0x34/0x48 + __queue_work+0x134/0x430 + queue_work_on+0x48/0x8c + timers_update_nohz+0x24/0x2c + tick_nohz_activate.isra.15.part.16+0x5c/0x80 + tick_setup_sched_timer+0xe0/0xf0 + hrtimer_run_queues+0x88/0xf8
So this is complaining that it cannot take pool->lock, which is WAIT_CONFIG while holding RCU, which presents a WAIT_CONFIG context.
This seems to implicate something is amiss, because that should be allowed. The thing it doesn't print is the context, which in the above case is a (hrtimer) interrupt.
I suspect this really is a hardirq context and the next patch won't cure things. It looks nohz (full?) related.
Frederic, can you untangle this?
---
kernel/locking/lockdep.c +31 -20
@@ -3952,10 +3952,36 @@ static int mark_lock(struct task_struct *curr, struct held_lock *this,return ret; } +static inline short task_wait_context (struct task_struct *curr) +{ +/* +* Set appropriate wait type for the context; for IRQs we have to take +* into account force_irqthread as that is implied by PREEMPT_RT. +*/ +if (curr->hardirq_context) { +/* +* Check if force_irqthreads will run us threaded. +*/ +if (curr->hardirq_threaded || curr->irq_config) +return LD_WAIT_CONFIG ; + +return LD_WAIT_SPIN ; + }else if (curr->softirq_context) { +/* +* Softirqs are always threaded. +*/ +return LD_WAIT_CONFIG ; + } + +return LD_WAIT_MAX ; +} +static int print_lock_invalid_wait_context (struct task_struct *curr,struct held_lock *hlock) { +short curr_inner; +if (!debug_locks_off ())return 0 ;if (debug_locks_silent)
@@ -3971,6 +3997,10 @@ print_lock_invalid_wait_context(struct task_struct *curr,print_lock (hlock);pr_warn ("other info that might help us debug this:\n" ); + + curr_inner =task_wait_context (curr); +pr_warn ("context-{%d:%d}\n" , curr_inner, curr_inner); +lockdep_print_held_locks (curr);pr_warn ("stack backtrace:\n" );
@@ -4017,26 +4047,7 @@ static int check_wait_context(struct task_struct *curr, struct held_lock *next) } depth++; -/* -* Set appropriate wait type for the context; for IRQs we have to take -* into account force_irqthread as that is implied by PREEMPT_RT. -*/ -if (curr->hardirq_context) { -/* -* Check if force_irqthreads will run us threaded. -*/ -if (curr->hardirq_threaded || curr->irq_config) - curr_inner =LD_WAIT_CONFIG ; -else - curr_inner =LD_WAIT_SPIN ; - }else if (curr->softirq_context) { -/* -* Softirqs are always threaded. -*/ - curr_inner =LD_WAIT_CONFIG ; - }else { - curr_inner =LD_WAIT_MAX ; - } + curr_inner =task_wait_context (curr);for (; depth < curr->lockdep_depth; depth++) {struct held_lock *prev = curr->held_locks + depth;
Peter Zijlstra
maintainerRe: [PATCH 6/9] lockdep: Introduce wait-type checks
Quoted 34 lines · Peter Zijlstra
Quoted 19 lines · Geert Uytterhoeven
On arm64 (e.g. R-Car H3 ES2.0):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679 Not tainted
+-----------------------------(quoted diff) +5
-0+swapper/5/0 is trying to lock: +ffffff86ff76f398 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x134/0x430 +other info that might help us debug this: +1 lock held by swapper/5/0: + #0: ffffffc01103a4a0 (rcu_read_lock){....}-{1:3}, at:rcu_lock_acquire.constprop.59+0x0/0x38
(quoted diff) +2
-0+stack backtrace: +CPU: 5 PID: 0 Comm: swapper/5 Not tainted5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679
(quoted diff) +4
-0+Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT) +Call trace: + dump_backtrace+0x0/0x180 + show_stack+0x14/0x1cReview
Quoted 10 lines
(quoted diff) +10
-0+ dump_stack+0xdc/0x12c + __lock_acquire+0x37c/0xf9c + lock_acquire+0x258/0x288 + _raw_spin_lock+0x34/0x48 + __queue_work+0x134/0x430 + queue_work_on+0x48/0x8c + timers_update_nohz+0x24/0x2c + tick_nohz_activate.isra.15.part.16+0x5c/0x80 + tick_setup_sched_timer+0xe0/0xf0 + hrtimer_run_queues+0x88/0xf8So this is complaining that it cannot take
pool->lock, which isWAIT_CONFIGwhile holding RCU, which presents aWAIT_CONFIGcontext.
Quoted 8 lines
This seems to implicate something is amiss, because that should be allowed. The thing it doesn't print is the context, which in the above case is a (hrtimer) interrupt.
I suspect this really is a hardirq context and the next patch won't cure things. It looks nohz (full?) related.
Frederic, can you untangle this?
Sebastian is right; I completely forgot the workqueue thing was still pending.
Frederic Weisbecker
maintainerRe: [PATCH 6/9] lockdep: Introduce wait-type checks
Quoted 39 lines · Peter Zijlstra
Quoted 38 lines · Peter Zijlstra
Quoted 19 lines · Geert Uytterhoeven
On arm64 (e.g. R-Car H3 ES2.0):
+=============================
+[ BUG: Invalid wait context ] +5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679 Not tainted
+-----------------------------(quoted diff) +5
-0+swapper/5/0 is trying to lock: +ffffff86ff76f398 (&pool->lock){..-.}-{3:3}, at: __queue_work+0x134/0x430 +other info that might help us debug this: +1 lock held by swapper/5/0: + #0: ffffffc01103a4a0 (rcu_read_lock){....}-{1:3}, at:rcu_lock_acquire.constprop.59+0x0/0x38
(quoted diff) +2
-0+stack backtrace: +CPU: 5 PID: 0 Comm: swapper/5 Not tainted5.6.0-salvator-x-09423-gb29514ba13a9c459-dirty #679
(quoted diff) +4
-0+Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT) +Call trace: + dump_backtrace+0x0/0x180 + show_stack+0x14/0x1cReview
Quoted 10 lines
(quoted diff) +10
-0+ dump_stack+0xdc/0x12c + __lock_acquire+0x37c/0xf9c + lock_acquire+0x258/0x288 + _raw_spin_lock+0x34/0x48 + __queue_work+0x134/0x430 + queue_work_on+0x48/0x8c + timers_update_nohz+0x24/0x2c + tick_nohz_activate.isra.15.part.16+0x5c/0x80 + tick_setup_sched_timer+0xe0/0xf0 + hrtimer_run_queues+0x88/0xf8So this is complaining that it cannot take
pool->lock, which isWAIT_CONFIGwhile holding RCU, which presents aWAIT_CONFIGcontext.This seems to implicate something is amiss, because that should be allowed. The thing it doesn't print is the context, which in the above case is a (hrtimer) interrupt.
Quoted 8 lines
Quoted 4 lines
I suspect this really is a hardirq context and the next patch won't cure things. It looks nohz (full?) related.
Frederic, can you untangle this?
Sebastian is right; I completely forgot the workqueue thing was still pending.
Ok good, because I had no better answer :)
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
In reply to Sebastian Andrzej SiewiorHello Peter and all,
I'm trying to understand the splat I've got with v6.18-rc1:
=============================
[ BUG: Invalid wait context ]
6.18.0-rc1+git476309497a53 #1 Tainted: G O
-----------------------------
some-user-space-process/1251 is trying to lock:ffff000005cbc548 (&counter->events_list_lock){....}-{3:3}, at: counter_push_event+0x68/0x430 [counter] other info that might help us debug this:
context-{2:2}
no locks held by some-user-space-process/1251.
stack backtrace: CPU: 0 UID: 0 PID: 1251 Comm: some-user-space-process Tainted: G O 6.18.0-rc1+git476309497a53 #1 PREEMPT Call trace: show_stack+0x20/0x38 (C) dump_stack_lvl+0x8c/0xd0 dump_stack+0x18/0x28 __lock_acquire+0x91c/0x1f78 lock_acquire+0x1c4/0x338 _raw_spin_lock_irqsave+0x68/0x98 counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398 handle_irq_event+0x54/0xb8 handle_simple_irq+0xe4/0x128 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gpio_irq_handler+0xa0/0x140 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gic_handle_irq+0x54/0x128 call_on_irq_stack+0x30/0x70 do_interrupt_handler+0x88/0x98 el0_interrupt+0x5c/0x270 __el0_irq_handler_common+0x18/0x28 el0t_64_irq_handler+0x10/0x20 el0t_64_irq+0x198/0x1a0
This is not PREEMPT_RT, but still CONFIG_PROVE_RAW_LOCK_NESTING=y config... The relevant logic in the current v6.18-rcX is still largely unchanged from the Subject commit and...
Quoted 69 lines · Sebastian Andrzej Siewior
From: Peter Zijlstra <[email protected]>
Extend lockdep to validate lock wait-type context.
The current wait-types are:
LD_WAIT_FREE, /* wait free, rcu etc.. */ LD_WAIT_SPIN, /* spin loops, raw_spinlock_t etc.. */ LD_WAIT_CONFIG, /* CONFIG_PREEMPT_LOCK, spinlock_t etc.. */ LD_WAIT_SLEEP, /* sleeping locks, mutex_t etc.. */Where lockdep validates that the current lock (the one being acquired) fits in the current wait-context (as generated by the held stack).
This ensures that there is no attempt to acquire mutexes while holding spinlocks, to acquire spinlocks while holding
raw_spinlocksand so on. In other words, its a more fancymight_sleep().Obviously RCU made the entire ordeal more complex than a simple single value test because RCU can be acquired in (pretty much) any context and while it presents a context to nested locks it is not the same as it got acquired in.
Therefore its necessary to split the
wait_typeinto two values, one representing the acquire (outer) and one representing the nested context (inner). For most 'normal' locks these two are the same.[ To make static initialization easier we have the rule that:
.outer == INV means .outer == .inner; because INV == 0. ]It further means that its required to find the minimal .inner of the held stack to compare against the outer of the new lock; because while 'normal' RCU presents a CONFIG type to nested locks, if it is taken while already holding a SPIN type it obviously doesn't relax the rules.
Below is an example output generated by the trivial test code:
raw_spin_lock(&foo); spin_lock(&bar); spin_unlock(&bar); raw_spin_unlock(&foo);[ BUG: Invalid wait context ]
-----------------------------
swapper/0/1 is trying to lock:
ffffc90000013f20(&bar){....}-{3:3}, at:kernel_init+0xdb/0x187other info that might help us debug this:
1 lock held by swapper/0/1:
#0:ffffc90000013ee0(&foo){+.+.}-{2:2}, at:kernel_init+0xd1/0x187The way to read it is to look at the new -{n,m} part in the lock description; -{3:3} for the attempted lock, and try and match that up to the held locks, which in this case is the one: -{2,2}.
This tells that the acquiring lock requires a more relaxed environment than presented by the lock stack.
Currently only the normal locks and RCU are converted, the rest of the lockdep users defaults to .inner = INV which is ignored. More conversions can be done when desired.
The check for
spinlock_tnesting is not enabled by default. It's a separate config option for now as there are known problems which are currently addressed. The config option allows to identify these problems and to verify that the solutions found are indeed solving them.The config switch will be removed and the checks will permanently enabled once the vast majority of issues has been addressed.
Quoted 7 lines
[ bigeasy: Move LD_WAIT_FREE,… out of
CONFIG_LOCKDEPto avoid compile
failure withCONFIG_DEBUG_SPINLOCK+ !CONFIG_LOCKDEP]
[ tglx: Add the config option ]Requested-by Thomas Gleixner <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Sebastian Andrzej Siewior <[email protected]>
[]
Quoted 75 lines
kernel/locking/lockdep.c +67
-0@@ -3798,6 +3806,113 @@ print_lock_nested_lock_not_held(struct task_struct *curr,dump_stack (); } +static int +print_lock_invalid_wait_context (struct task_struct *curr, +struct held_lock *hlock) +{ +if (!debug_locks_off ()) +return 0 ; +if (debug_locks_silent) +return 0 ; + +pr_warn ("\n" ); +pr_warn ("=============================\n" ); +pr_warn ("[ BUG: Invalid wait context ]\n" ); +print_kernel_ident (); +pr_warn ("-----------------------------\n" ); + +pr_warn ("%s/%d is trying to lock:\n" , curr->comm,task_pid_nr (curr)); +print_lock (hlock); + +pr_warn ("other info that might help us debug this:\n" ); +lockdep_print_held_locks (curr); + +pr_warn ("stack backtrace:\n" ); +dump_stack (); + +return 0 ; +} + +/* +* Verify the wait_type context. +* +* This check validates we takes locks in the right wait-type order; that is it +* ensures that we do not take mutexes inside spinlocks and do not attempt to +* acquire spinlocks inside raw_spinlocks and the sort. +* +* The entire thing is slightly more complex because of RCU, RCU is a lock that +* can be taken from (pretty much) any context but also has constraints. +* However when taken in a stricter environment the RCU lock does not loosen +* the constraints. +* +* Therefore we must look for the strictest environment in the lock stack and +* compare that to the lock we're trying to acquire. +*/ +static int check_wait_context (struct task_struct *curr,struct held_lock *next) +{ +short next_inner =hlock_class (next)->wait_type_inner; +short next_outer =hlock_class (next)->wait_type_outer; +short curr_inner; +int depth; + +if (!curr->lockdep_depth || !next_inner || next->trylock) +return 0 ; + +if (!next_outer) + next_outer = next_inner; + +/* +* Find start of current irq_context.. +*/ +for (depth = curr->lockdep_depth -1 ; depth >=0 ; depth--) { +struct held_lock *prev = curr->held_locks + depth; +if (prev->irq_context != next->irq_context) +break ; + } + depth++; + +/* +* Set appropriate wait type for the context; for IRQs we have to take
kernel/locking/lockdep.c print_lock_nested_lock_not_held(struct task_struct *curr, · line 3885
Quoted 10 lines
kernel/locking/lockdep.c +10
-0+* into account force_irqthread as that is implied by PREEMPT_RT. +*/ +if (curr->hardirq_context) { +/* +* Check if force_irqthreads will run us threaded. +*/ +if (curr->hardirq_threaded) + curr_inner =LD_WAIT_CONFIG ; +else + curr_inner =LD_WAIT_SPIN ;
... even though it's "if (curr->hardirq_threaded || curr->irq_config)" in the meanwhile, it is unclear to me, why is it conditional at all and not always "LD_WAIT_CONFIG" in hardirq context?
With !PREEMPT_RT but CONFIG_PROVE_RAW_LOCK_NESTING=y we always end up with LD_WAIT_SPIN here and it always warns if we take "spinlock_t" in hardirq context (as in the splat above). But the latter should be legal under all circumstances, isn't it?
Quoted 33 lines
kernel/locking/lockdep.c +30
-0+ }else if (curr->softirq_context) { +/* +* Softirqs are always threaded. +*/ + curr_inner =LD_WAIT_CONFIG ; + }else { + curr_inner =LD_WAIT_MAX ; + } + +for (; depth < curr->lockdep_depth; depth++) { +struct held_lock *prev = curr->held_locks + depth; +short prev_inner =hlock_class (prev)->wait_type_inner; + +if (prev_inner) { +/* +* We can have a bigger inner than a previous one +* when outer is smaller than inner, as with RCU. +* +* Also due to trylocks. +*/ + curr_inner =min (curr_inner, prev_inner); + } + } + +if (next_outer > curr_inner) +return print_lock_invalid_wait_context (curr, next); + +return 0 ; +} +static int __lock_is_held (const struct lockdep_map *lock,int read);/*
Signature · 3 lines
Alexander Sverdlin Siemens AG www.siemens.com
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Quoted 40 lines · Alexander
Hello Peter and all,
I'm trying to understand the splat I've got with v6.18-rc1:
=============================
[ BUG: Invalid wait context ]
6.18.0-rc1+git476309497a53 #1 Tainted: G O
-----------------------------
some-user-space-process/1251 is trying to lock:ffff000005cbc548(&counter->events_list_lock){....}-{3:3}, at:counter_push_event+0x68/0x430[counter] other info that might help us debug this:
context-{2:2}
no locks held by some-user-space-process/1251.stack backtrace: CPU: 0 UID: 0 PID: 1251 Comm: some-user-space-process Tainted: G O 6.18.0-rc1+git476309497a53 #1 PREEMPT Call trace: show_stack+0x20/0x38 (C) dump_stack_lvl+0x8c/0xd0 dump_stack+0x18/0x28 __lock_acquire+0x91c/0x1f78 lock_acquire+0x1c4/0x338 _raw_spin_lock_irqsave+0x68/0x98 counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398 handle_irq_event+0x54/0xb8 handle_simple_irq+0xe4/0x128 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gpio_irq_handler+0xa0/0x140 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gic_handle_irq+0x54/0x128 call_on_irq_stack+0x30/0x70 do_interrupt_handler+0x88/0x98 el0_interrupt+0x5c/0x270 __el0_irq_handler_common+0x18/0x28 el0t_64_irq_handler+0x10/0x20 el0t_64_irq+0x198/0x1a0
Quoted 4 lines
This is not
PREEMPT_RT, but stillCONFIG_PROVE_RAW_LOCK_NESTING=yconfig... The relevant logic in the current v6.18-rcX is still largely unchanged from the Subject commit and...
The validation is not limited to PREEMPT_RT enabled. …
Quoted 4 lines
Quoted 4 lines
kernel/locking/lockdep.c +0
-0
…
Quoted 16 lines
kernel/locking/lockdep.c print_lock_nested_lock_not_held(struct task_struct *curr, · line 3885
Quoted 11 lines
kernel/locking/lockdep.c +11
-0+* Set appropriate wait type for the context; for IRQs we have to take +* into account force_irqthread as that is implied by PREEMPT_RT. +*/ +if (curr->hardirq_context) { +/* +* Check if force_irqthreads will run us threaded. +*/ +if (curr->hardirq_threaded) + curr_inner =LD_WAIT_CONFIG ; +else + curr_inner =LD_WAIT_SPIN ;... even though it's "if (
curr->hardirq_threaded||curr->irq_config)" in the meanwhile, it is unclear to me, why is it conditional at all and not always "LD_WAIT_CONFIG" in hardirq context?
Quoted 4 lines
With !PREEMPT_RT but
CONFIG_PROVE_RAW_LOCK_NESTING=ywe always end up withLD_WAIT_SPINhere and it always warns if we take "spinlock_t" in hardirq context (as in the splat above). But the latter should be legal under all circumstances, isn't it?
Nope. The validation is performed always. The architectures which don't support PREEMPT_RT are excluded for $reasons that is why PROVE_RAW_LOCK_NESTING remains an option there.
A spinlock_t is a sleeping lock in a PREEMPT_RT context and the interrupt hander are threaded. In this case a spinlock_t is okay. However, if a spinlock_t is used in a non-forced-threaded environment (such as in a primary handler with IRQF_NO_THREAD) then this handler will not be threaded. Therefore spinlock_t is not a valid class.
Sebastian
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Hello Sebastian,
Quoted 71 lines · On Mon, 2025-11-17 at 14:55 +0100, [email protected] wrote:
Quoted 35 lines
[ BUG: Invalid wait context ]
6.18.0-rc1+git476309497a53 #1 Tainted: G O
-----------------------------
some-user-space-process/1251 is trying to lock:ffff000005cbc548(&counter->events_list_lock){....}-{3:3}, at:counter_push_event+0x68/0x430[counter] other info that might help us debug this:
context-{2:2}
no locks held by some-user-space-process/1251.stack backtrace: CPU: 0 UID: 0 PID: 1251 Comm: some-user-space-process Tainted: G O 6.18.0-rc1+git476309497a53 #1 PREEMPT Call trace: show_stack+0x20/0x38 (C) dump_stack_lvl+0x8c/0xd0 dump_stack+0x18/0x28 __lock_acquire+0x91c/0x1f78 lock_acquire+0x1c4/0x338 _raw_spin_lock_irqsave+0x68/0x98 counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398 handle_irq_event+0x54/0xb8 handle_simple_irq+0xe4/0x128 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gpio_irq_handler+0xa0/0x140 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gic_handle_irq+0x54/0x128 call_on_irq_stack+0x30/0x70 do_interrupt_handler+0x88/0x98 el0_interrupt+0x5c/0x270 __el0_irq_handler_common+0x18/0x28 el0t_64_irq_handler+0x10/0x20 el0t_64_irq+0x198/0x1a0Quoted 4 lines
This is not
PREEMPT_RT, but stillCONFIG_PROVE_RAW_LOCK_NESTING=yconfig... The relevant logic in the current v6.18-rcX is still largely unchanged from the Subject commit and...The validation is not limited to
PREEMPT_RTenabled. …Quoted 4 lines
Quoted 4 lines
kernel/locking/lockdep.c +0
-0…
Quoted 16 lines
kernel/locking/lockdep.c print_lock_nested_lock_not_held(struct task_struct *curr, · line 3885
Quoted 11 lines
kernel/locking/lockdep.c +11
-0+* Set appropriate wait type for the context; for IRQs we have to take +* into account force_irqthread as that is implied by PREEMPT_RT. +*/ +if (curr->hardirq_context) { +/* +* Check if force_irqthreads will run us threaded. +*/ +if (curr->hardirq_threaded) + curr_inner =LD_WAIT_CONFIG ; +else + curr_inner =LD_WAIT_SPIN ;... even though it's "if (
curr->hardirq_threaded||curr->irq_config)" in the meanwhile, it is unclear to me, why is it conditional at all and not always "LD_WAIT_CONFIG" in hardirq context?Quoted 4 lines
With !PREEMPT_RT but
CONFIG_PROVE_RAW_LOCK_NESTING=ywe always end up withLD_WAIT_SPINhere and it always warns if we take "spinlock_t" in hardirq context (as in the splat above). But the latter should be legal under all circumstances, isn't it?Nope. The validation is performed always. The architectures which don't support
PREEMPT_RTare excluded for $reasons that is whyPROVE_RAW_LOCK_NESTINGremains an option there.
Quoted 5 lines
A
spinlock_tis a sleeping lock in aPREEMPT_RTcontext and the interrupt hander are threaded. In this case aspinlock_tis okay. However, if aspinlock_tis used in a non-forced-threaded environment (such as in a primary handler withIRQF_NO_THREAD) then this handler will not be threaded. Thereforespinlock_tis not a valid class.
I'm not in PREEMPT_RT, but now CONFIG_PROVE_RAW_LOCK_NESTING=y is forced for ARM64. And as I'm not in PREEMPT_RT, it's allowed to take spinlock_t in an IRQ, no matter what (that's what happens in the above splat, I believe).
So does it mean, that LOCKDEP is not usable any longer on ARM64 (for !PREEMPT_RT kernels)?
Signature · 3 lines
Alexander Sverdlin Siemens AG www.siemens.com
Paul E. McKenney
maintainerRe: [PATCH 6/9] lockdep: Introduce wait-type checks
Quoted 80 lines · Alexander
Hello Sebastian,
Quoted 71 lines · On Mon, 2025-11-17 at 14:55 +0100, [email protected] wrote:
Quoted 35 lines
[ BUG: Invalid wait context ]
6.18.0-rc1+git476309497a53 #1 Tainted: G O
-----------------------------
some-user-space-process/1251 is trying to lock:ffff000005cbc548(&counter->events_list_lock){....}-{3:3}, at:counter_push_event+0x68/0x430[counter] other info that might help us debug this:
context-{2:2}
no locks held by some-user-space-process/1251.stack backtrace: CPU: 0 UID: 0 PID: 1251 Comm: some-user-space-process Tainted: G O 6.18.0-rc1+git476309497a53 #1 PREEMPT Call trace: show_stack+0x20/0x38 (C) dump_stack_lvl+0x8c/0xd0 dump_stack+0x18/0x28 __lock_acquire+0x91c/0x1f78 lock_acquire+0x1c4/0x338 _raw_spin_lock_irqsave+0x68/0x98 counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398 handle_irq_event+0x54/0xb8 handle_simple_irq+0xe4/0x128 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gpio_irq_handler+0xa0/0x140 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gic_handle_irq+0x54/0x128 call_on_irq_stack+0x30/0x70 do_interrupt_handler+0x88/0x98 el0_interrupt+0x5c/0x270 __el0_irq_handler_common+0x18/0x28 el0t_64_irq_handler+0x10/0x20 el0t_64_irq+0x198/0x1a0Quoted 4 lines
This is not
PREEMPT_RT, but stillCONFIG_PROVE_RAW_LOCK_NESTING=yconfig... The relevant logic in the current v6.18-rcX is still largely unchanged from the Subject commit and...The validation is not limited to
PREEMPT_RTenabled. …Quoted 4 lines
Quoted 4 lines
kernel/locking/lockdep.c +0
-0…
Quoted 16 lines
kernel/locking/lockdep.c print_lock_nested_lock_not_held(struct task_struct *curr, · line 3885
Quoted 11 lines
kernel/locking/lockdep.c +11
-0+* Set appropriate wait type for the context; for IRQs we have to take +* into account force_irqthread as that is implied by PREEMPT_RT. +*/ +if (curr->hardirq_context) { +/* +* Check if force_irqthreads will run us threaded. +*/ +if (curr->hardirq_threaded) + curr_inner =LD_WAIT_CONFIG ; +else + curr_inner =LD_WAIT_SPIN ;... even though it's "if (
curr->hardirq_threaded||curr->irq_config)" in the meanwhile, it is unclear to me, why is it conditional at all and not always "LD_WAIT_CONFIG" in hardirq context?Quoted 4 lines
With !PREEMPT_RT but
CONFIG_PROVE_RAW_LOCK_NESTING=ywe always end up withLD_WAIT_SPINhere and it always warns if we take "spinlock_t" in hardirq context (as in the splat above). But the latter should be legal under all circumstances, isn't it?Nope. The validation is performed always. The architectures which don't support
PREEMPT_RTare excluded for $reasons that is whyPROVE_RAW_LOCK_NESTINGremains an option there.Quoted 5 lines
A
spinlock_tis a sleeping lock in aPREEMPT_RTcontext and the interrupt hander are threaded. In this case aspinlock_tis okay. However, if aspinlock_tis used in a non-forced-threaded environment (such as in a primary handler withIRQF_NO_THREAD) then this handler will not be threaded. Thereforespinlock_tis not a valid class.
Quoted 7 lines
I'm not in
PREEMPT_RT, but nowCONFIG_PROVE_RAW_LOCK_NESTING=yis forced for ARM64. And as I'm not inPREEMPT_RT, it's allowed to takespinlock_tin an IRQ, no matter what (that's what happens in the above splat, I believe).So does it mean, that LOCKDEP is not usable any longer on ARM64 (for !PREEMPT_RT kernels)?
LOCKDEP is quite usable, and in this case it is just doing its job by preventing people from inadvertently breaking RT. Which arm64 really does support, as you can see by searching for "select ARCH_SUPPORTS_RT" in arch/arm64/Kconfig. So even arm64-specific code must follow the PREEMPT_RT rules, which lockdep is enforcing.
So how to fix this? One option is to use a raw_spinlock_t instead of a spinlock_t. This works because a raw_spinlock_t remains a spinlock in RT, but it also restricts what you can do while holding the lock.
Does that help?
Thanx, Paul
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Hi Paul,
Quoted 91 lines · Paul E. McKenney
Quoted 77 lines
Quoted 71 lines
Quoted 35 lines
[ BUG: Invalid wait context ]
6.18.0-rc1+git476309497a53 #1 Tainted: G O
-----------------------------
some-user-space-process/1251 is trying to lock:ffff000005cbc548(&counter->events_list_lock){....}-{3:3}, at:counter_push_event+0x68/0x430[counter] other info that might help us debug this:
context-{2:2}
no locks held by some-user-space-process/1251.stack backtrace: CPU: 0 UID: 0 PID: 1251 Comm: some-user-space-process Tainted: G O 6.18.0-rc1+git476309497a53 #1 PREEMPT Call trace: show_stack+0x20/0x38 (C) dump_stack_lvl+0x8c/0xd0 dump_stack+0x18/0x28 __lock_acquire+0x91c/0x1f78 lock_acquire+0x1c4/0x338 _raw_spin_lock_irqsave+0x68/0x98 counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398 handle_irq_event+0x54/0xb8 handle_simple_irq+0xe4/0x128 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gpio_irq_handler+0xa0/0x140 handle_irq_desc+0x48/0x68 generic_handle_domain_irq+0x24/0x38 gic_handle_irq+0x54/0x128 call_on_irq_stack+0x30/0x70 do_interrupt_handler+0x88/0x98 el0_interrupt+0x5c/0x270 __el0_irq_handler_common+0x18/0x28 el0t_64_irq_handler+0x10/0x20 el0t_64_irq+0x198/0x1a0Quoted 4 lines
This is not
PREEMPT_RT, but stillCONFIG_PROVE_RAW_LOCK_NESTING=yconfig... The relevant logic in the current v6.18-rcX is still largely unchanged from the Subject commit and...The validation is not limited to
PREEMPT_RTenabled. …Quoted 4 lines
Quoted 4 lines
kernel/locking/lockdep.c +0
-0…
Quoted 16 lines
kernel/locking/lockdep.c print_lock_nested_lock_not_held(struct task_struct *curr, · line 3885
Quoted 11 lines
kernel/locking/lockdep.c +11
-0+* Set appropriate wait type for the context; for IRQs we have to take +* into account force_irqthread as that is implied by PREEMPT_RT. +*/ +if (curr->hardirq_context) { +/* +* Check if force_irqthreads will run us threaded. +*/ +if (curr->hardirq_threaded) + curr_inner =LD_WAIT_CONFIG ; +else + curr_inner =LD_WAIT_SPIN ;... even though it's "if (
curr->hardirq_threaded||curr->irq_config)" in the meanwhile, it is unclear to me, why is it conditional at all and not always "LD_WAIT_CONFIG" in hardirq context?Quoted 4 lines
With !PREEMPT_RT but
CONFIG_PROVE_RAW_LOCK_NESTING=ywe always end up withLD_WAIT_SPINhere and it always warns if we take "spinlock_t" in hardirq context (as in the splat above). But the latter should be legal under all circumstances, isn't it?Nope. The validation is performed always. The architectures which don't support
PREEMPT_RTare excluded for $reasons that is whyPROVE_RAW_LOCK_NESTINGremains an option there.Quoted 5 lines
A
spinlock_tis a sleeping lock in aPREEMPT_RTcontext and the interrupt hander are threaded. In this case aspinlock_tis okay. However, if aspinlock_tis used in a non-forced-threaded environment (such as in a primary handler withIRQF_NO_THREAD) then this handler will not be threaded. Thereforespinlock_tis not a valid class.Quoted 7 lines
I'm not in
PREEMPT_RT, but nowCONFIG_PROVE_RAW_LOCK_NESTING=yis forced for ARM64. And as I'm not inPREEMPT_RT, it's allowed to takespinlock_tin an IRQ, no matter what (that's what happens in the above splat, I believe).So does it mean, that LOCKDEP is not usable any longer on ARM64 (for !PREEMPT_RT kernels)?
LOCKDEP is quite usable, and in this case it is just doing its job by preventing people from inadvertently breaking RT. Which arm64 really does support, as you can see by searching for "select
ARCH_SUPPORTS_RT" in arch/arm64/Kconfig. So even arm64-specific code must follow thePREEMPT_RTrules, which lockdep is enforcing.
Quoted 5 lines
So how to fix this? One option is to use a
raw_spinlock_tinstead of a spinlock_t. This works because araw_spinlock_tremains a spinlock in RT, but it also restricts what you can do while holding the lock.Does that help?
Mmm. Not yet... Does this mean we are not allowed to use "spinlock_t" in IRQs any longer? Where has this been announced? This probably means that the majority of IRQ handlers were already converted tree-wide?
This also would mean that spin_lock_irqsave() shall not exist at all so that we are forced to use raw_spin_lock_irqsave()?
The current documentation says [1]:
"Use raw_spinlock_t only in real critical core code, low-level interrupt handling and places where disabling preemption or interrupts is required, for example, to safely access hardware state."
So does the counter_push_event() constitute such a code?
[1] https://docs.kernel.org/locking/locktypes.html#raw-spinlock-t-and-spinlock-t
Signature · 3 lines
Alexander Sverdlin Siemens AG www.siemens.com
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Quoted 1 line · Alexander
Hi Paul,
Hi Alexander,
…
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Call trace:
…
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398
…
Quoted 8 lines
Quoted 1 line
Does that help?
Mmm. Not yet... Does this mean we are not allowed to use "
spinlock_t" in IRQs any longer? Where has this been announced? This probably means that the majority of IRQ handlers were already converted tree-wide?This also would mean that
spin_lock_irqsave()shall not exist at all so that we are forced to useraw_spin_lock_irqsave()?
As I tried to explain, if you use IRQF_NO_THREAD then the handler will not be threaded. Looking at drivers/counter/interrupt-cnt.c, it does
| irq_set_status_flags(priv->irq, IRQ_NOAUTOEN); | ret = devm_request_irq(dev, priv->irq, interrupt_cnt_isr, | IRQF_TRIGGER_RISING | IRQF_NO_THREAD, | dev_name(dev), counter); | if (ret)
So the irq-core does not thread the handler as requested. So it must not use spinlock_t in such a case.
And judging from the code, that IRQF_NO_THREAD should be removed. Not sure why you are the only seeing that splat.
Sebastian
Re: [PATCH 6/9] lockdep: Introduce wait-type checks
Hi Sebstian,
Quoted 22 lines · On Mon, 2025-11-17 at 16:13 +0100, [email protected] wrote:
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
counter_push_event+0x68/0x430 [counter] interrupt_cnt_isr+0x40/0x78 [interrupt_cnt] __handle_irq_event_percpu+0xa4/0x398…
Quoted 8 lines
Quoted 1 line
Does that help?
Mmm. Not yet... Does this mean we are not allowed to use "
spinlock_t" in IRQs any longer? Where has this been announced? This probably means that the majority of IRQ handlers were already converted tree-wide?This also would mean that
spin_lock_irqsave()shall not exist at all so that we are forced to useraw_spin_lock_irqsave()?As I tried to explain, if you use
IRQF_NO_THREADthen the handler will not be threaded. Looking atdrivers/counter/interrupt-cnt.c, it doesQuoted 5 lines
irq_set_status_flags(priv->irq, IRQ_NOAUTOEN); ret =devm_request_irq(dev, priv->irq, interrupt_cnt_isr, IRQF_TRIGGER_RISING | IRQF_NO_THREAD, dev_name(dev), counter); if (ret)
Quoted 4 lines
So the irq-core does not thread the handler as requested. So it must not use
spinlock_tin such a case.
And judging from the code, thatIRQF_NO_THREADshould be removed. Not sure why you are the only seeing that splat.
thanks for looking into that! I've totally missed IRQF_NO_THREAD! I'll post a patch removing it...
Signature · 3 lines
Alexander Sverdlin Siemens AG www.siemens.com
PATCH 7/9 lockdep: Add hrtimer context tracing bits
In reply to Sebastian Andrzej SiewiorSet current->irq_config = 1 for hrtimers which are not marked to expire in hard interrupt context during hrtimer_init(). These timers will expire in softirq context on PREEMPT_RT.
Setting this allows lockdep to differentiate these timers. If a timer is marked to expire in hard interrupt context then the timer callback is not supposed to acquire a regular spinlock instead of a raw_spinlock in the expiry callback.
--- include/linux/irqflags.h | 15 +++++++++++++++ include/linux/sched.h | 1 + kernel/locking/lockdep.c | 2 +- kernel/time/hrtimer.c | 6 +++++- 4 files changed, 22 insertions(+), 2 deletions(-)
include/linux/irqflags.h +15 -0
@@ -56,6 +56,19 @@ do { \do { \ current->softirq_context--; \ } while (0) + +# define lockdep_hrtimer_enter (__hrtimer) \ + do { \ + if (!__hrtimer->is_hard) \ + current->irq_config = 1; \ + } while (0) + +# define lockdep_hrtimer_exit (__hrtimer) \ + do { \ + if (!__hrtimer->is_hard) \ + current->irq_config = 0; \ + } while (0) +#else # define trace_hardirqs_on () do { } while (0)# define trace_hardirqs_off () do { } while (0)
@@ -68,6 +81,8 @@ do { \# define trace_hardirq_exit () do { } while (0)# define lockdep_softirq_enter () do { } while (0)# define lockdep_softirq_exit () do { } while (0) +# define lockdep_hrtimer_enter (__hrtimer) do { } while (0) +# define lockdep_hrtimer_exit (__hrtimer) do { } while (0)#endif #if defined(CONFIG_IRQSOFF_TRACER) || \
include/linux/sched.h +1 -0
@@ -983,6 +983,7 @@ struct task_struct {unsigned int softirq_enable_event;int softirqs_enabled;int softirq_context; +int irq_config;#endif #ifdef CONFIG_LOCKDEP
kernel/locking/lockdep.c +1 -1
@@ -3879,7 +3879,7 @@ static int check_wait_context(struct task_struct *curr, struct held_lock *next)/* * Check if force_irqthreads will run us threaded. */ -if (curr->hardirq_threaded) +if (curr->hardirq_threaded || curr->irq_config) curr_inner =LD_WAIT_CONFIG ;else curr_inner =LD_WAIT_SPIN ;
kernel/time/hrtimer.c +5 -1
@@ -1404,7 +1404,7 @@ static void __hrtimer_init(struct hrtimer *timer, clockid_t clock_id, base = softtimer ?HRTIMER_MAX_CLOCK_BASES /2 :0 ; base +=hrtimer_clockid_to_base (clock_id); timer->is_soft = softtimer; - timer->is_hard = !softtimer; + timer->is_hard = !!(mode &HRTIMER_MODE_HARD ); timer->base = &cpu_base->clock_base[base];timerqueue_init (&timer->node); }
@@ -1514,7 +1514,11 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base,*/ raw_spin_unlock_irqrestore (&cpu_base->lock, flags);trace_hrtimer_expire_entry (timer, now); +lockdep_hrtimer_enter (timer); + restart =fn (timer); + +lockdep_hrtimer_exit (timer);trace_hrtimer_expire_exit (timer);raw_spin_lock_irq (&cpu_base->lock);
Signature · 1 line
2.25.1
PATCH 8/9 lockdep: Annotate irq_work
In reply to Sebastian Andrzej SiewiorMark irq_work items with IRQ_WORK_HARD_IRQ which should be invoked in hardirq context even on PREEMPT_RT. IRQ_WORK without this flag will be invoked in softirq context on PREEMPT_RT.
Set ->irq_config to 1 for the IRQ_WORK items which are invoked in softirq context so lockdep knows that these can safely acquire a spinlock_t.
--- include/linux/irq_work.h | 2 ++ include/linux/irqflags.h | 13 +++++++++++++ kernel/irq_work.c | 2 ++ kernel/rcu/tree.c | 1 + kernel/time/tick-sched.c | 1 + 5 files changed, 19 insertions(+)
include/linux/irq_work.h +2 -0
@@ -18,6 +18,8 @@/* Doesn't want IPI, wait for tick: */ #define IRQ_WORK_LAZY BIT(2) +/* Run hard IRQ context, even on RT */ +#define IRQ_WORK_HARD_IRQ BIT(3)#define IRQ_WORK_CLAIMED (IRQ_WORK_PENDING | IRQ_WORK_BUSY)
include/linux/irqflags.h +13 -0
@@ -69,6 +69,17 @@ do { \ current->irq_config =0 ; \ } while (0) +# define lockdep_irq_work_enter (__work) \ + do { \ + if (!(atomic_read(&__work->flags) & IRQ_WORK_HARD_IRQ))\ + current->irq_config = 1; \ + } while (0) +# define lockdep_irq_work_exit (__work) \ + do { \ + if (!(atomic_read(&__work->flags) & IRQ_WORK_HARD_IRQ))\ + current->irq_config = 0; \ + } while (0) +#else # define trace_hardirqs_on () do { } while (0)# define trace_hardirqs_off () do { } while (0)
@@ -83,6 +94,8 @@ do { \# define lockdep_softirq_exit () do { } while (0)# define lockdep_hrtimer_enter (__hrtimer) do { } while (0)# define lockdep_hrtimer_exit (__hrtimer) do { } while (0) +# define lockdep_irq_work_enter (__work) do { } while (0) +# define lockdep_irq_work_exit (__work) do { } while (0)#endif #if defined(CONFIG_IRQSOFF_TRACER) || \
kernel/irq_work.c +2 -0
@@ -153,7 +153,9 @@ static void irq_work_run_list(struct llist_head *list)*/ flags =atomic_fetch_andnot (IRQ_WORK_PENDING , &work->flags); +lockdep_irq_work_enter (work); work->func (work); +lockdep_irq_work_exit (work);/* * Clear the BUSY bit and return to the free state if * no-one else claimed it meanwhile.
kernel/rcu/tree.c +1 -0
@@ -1113,6 +1113,7 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp) !rdp->rcu_iw_pending && rdp->rcu_iw_gp_seq != rnp->gp_seq && (rnp->ffmask & rdp->grpmask)) {init_irq_work (&rdp->rcu_iw, rcu_iw_handler); +atomic_set (&rdp->rcu_iw.flags,IRQ_WORK_HARD_IRQ ); rdp->rcu_iw_pending = true; rdp->rcu_iw_gp_seq = rnp->gp_seq;irq_work_queue_on (&rdp->rcu_iw, rdp->cpu);
kernel/time/tick-sched.c +1 -0
@@ -245,6 +245,7 @@ static void nohz_full_kick_func(struct irq_work *work) static DEFINE_PER_CPU(struct irq_work, nohz_full_kick_work) = { .func = nohz_full_kick_func, + .flags =ATOMIC_INIT (IRQ_WORK_HARD_IRQ ), };/*
Signature · 1 line
2.25.1
PATCH 9/9 lockdep: Add posixtimer context tracing bits
In reply to Sebastian Andrzej SiewiorSplitting run_posix_cpu_timers() into two parts is work in progress which is stuck on other entry code related problems. The heavy lifting which involves locking of sighand lock will be moved into task context so the necessary execution time is burdened on the task and not on interrupt context.
Until this work completes lockdep with the spinlock nesting rules enabled would emit warnings for this known context.
Prevent it by setting "->irq_config = 1" for the invocation of run_posix_cpu_timers() so lockdep does not complain when sighand lock is acquried. This will be removed once the split is completed.
--- include/linux/irqflags.h | 12 ++++++++++++ kernel/time/posix-cpu-timers.c | 6 +++++- 2 files changed, 17 insertions(+), 1 deletion(-)
include/linux/irqflags.h +12 -0
@@ -69,6 +69,16 @@ do { \ current->irq_config =0 ; \ } while (0) +# define lockdep_posixtimer_enter () \ + do { \ + current->irq_config = 1; \ + } while (0) + +# define lockdep_posixtimer_exit () \ + do { \ + current->irq_config = 0; \ + } while (0) +# define lockdep_irq_work_enter (__work) \ do { \ if (!(atomic_read(&__work->flags) & IRQ_WORK_HARD_IRQ))\
@@ -94,6 +104,8 @@ do { \# define lockdep_softirq_exit () do { } while (0)# define lockdep_hrtimer_enter (__hrtimer) do { } while (0)# define lockdep_hrtimer_exit (__hrtimer) do { } while (0) +# define lockdep_posixtimer_enter () do { } while (0) +# define lockdep_posixtimer_exit () do { } while (0)# define lockdep_irq_work_enter (__work) do { } while (0)# define lockdep_irq_work_exit (__work) do { } while (0)#endif
kernel/time/posix-cpu-timers.c +5 -1
@@ -1126,8 +1126,11 @@ void run_posix_cpu_timers(void)if (!fastpath_timer_check (tsk))return ; -if (!lock_task_sighand (tsk, &flags)) +lockdep_posixtimer_enter (); +if (!lock_task_sighand (tsk, &flags)) { +lockdep_posixtimer_exit ();return ; + }/* * Here we take off tsk->signal->cpu_timers[N] and * tsk->cpu_timers[N] all the timers that are firing, and
@@ -1169,6 +1172,7 @@ void run_posix_cpu_timers(void)cpu_timer_fire (timer);spin_unlock (&timer->it_lock); } +lockdep_posixtimer_exit (); }/*
Signature · 1 line
2.25.1