DEPT(Dependency Tracker)
Subsystems (21)
v9 cover DEPT(Dependency Tracker)
I'm happy to see that DEPT reports a real problem in practice. See:
https://lore.kernel.org/lkml/[email protected]/#t
https://lore.kernel.org/lkml/[email protected]/
Nevertheless, I apologize for the lack of document. I promise to add it before it gets needed to use DEPT's APIs by users. For now, you can use DEPT just with CONFIG_DEPT on.
---
Hi Linus and folks,
I've been developing a tool for detecting deadlock possibilities by tracking wait/event rather than lock(?) acquisition order to try to cover all synchonization machanisms. It's done on v6.2-rc2.
https://github.com/lgebyungchulpark/linux-dept/commits/dept2.4_on_v6.2-rc2
Benifit:
0. Works with all lock primitives.
1. Works with wait_for_completion()/complete().
2. Works with 'wait' on PG_locked.
3. Works with 'wait' on PG_writeback.
4. Works with swait/wakeup.
5. Works with waitqueue.
6. Works with wait_bit.
7. Multiple reports are allowed.
8. Deduplication control on multiple reports.
9. Withstand false positives thanks to 6.
10. Easy to tag any wait/event.
Future work:
0. To make it more stable.
1. To separates Dept from Lockdep.
2. To improves performance in terms of time and space.
3. To use Dept as a dependency engine for Lockdep.
4. To add any missing tags of wait/event in the kernel.
5. To deduplicate stack trace.
How to interpret reports:
1. E(event) in each context cannot be triggered because of the W(wait) that cannot be woken.
2. The stack trace helping find the problematic code is located in each conext's detail.
Thanks,
Byungchul
---
Changes from v8:
1. Fix build error by adding EXPORT_SYMBOL(PG_locked_map) and EXPORT_SYMBOL(PG_writeback_map) for kernel module build - appologize for that. (reported by kernel test robot)
2. Fix build error by removing header file's circular dependency that was caused by "atomic.h", "kernel.h" and "irqflags.h", which I introduced - appolgize for that. (reported by kernel test robot)
Changes from v7:
1. Fix a bug that cannot track rwlock dependency properly, introduced in v7. (reported by Boqun and lockdep selftest)
2. Track wait/event of PG_{locked,writeback} more aggressively assuming that when a bit of PG_{locked,writeback} is cleared there might be waits on the bit. (reported by Linus, Hillf and syzbot)
3. Fix and clean bad style code e.i. unnecessarily introduced a randome pattern and so on. (pointed out by Linux)
4. Clean code for applying DEPT to wait_for_completion().
Changes from v6:
1. Tie to task scheduler code to track sleep and try_to_wake_up() assuming sleeps cause waits, try_to_wake_up()s would be the events that those are waiting for, of course with proper DEPT annotations, sdt_might_sleep_weak(), sdt_might_sleep_strong() and so on. For these cases, class is classified at sleep entrance rather than the synchronization initialization code. Which would extremely reduce false alarms.
2. Remove the DEPT associated instance in each page struct for tracking dependencies by PG_locked and PG_writeback thanks to the 1. work above.
3. Introduce CONFIG_DEPT_AGGRESIVE_TIMEOUT_WAIT to suppress reports that waits with timeout set are involved, for those who don't like verbose reporting.
4. Add a mechanism to refill the internal memory pools on running out so that DEPT could keep working as long as free memory is available in the system.
5. Re-enable tracking hashed-waitqueue wait. That's going to no longer generate false positives because class is classified at sleep entrance rather than the waitqueue initailization.
6. Refactor to make it easier to port onto each new version of the kernel.
7. Apply DEPT to dma fence.
8. Do trivial optimizaitions.
Changes from v5:
1. Use just pr_warn_once() rather than WARN_ONCE() on the lack of internal resources because WARN_*() printing stacktrace is too much for informing the lack. (feedback from Ted, Hyeonggon)
2. Fix trivial bugs like missing initializing a struct before using it.
3. Assign a different class per task when handling onstack variables for waitqueue or the like. Which makes Dept distinguish between onstack variables of different tasks so as to prevent false positives. (reported by Hyeonggon)
4. Make Dept aware of even raw_local_irq_*() to prevent false positives. (reported by Hyeonggon)
5. Don't consider dependencies between the events that might be triggered within __schedule() and the waits that requires __schedule(), real ones. (reported by Hyeonggon)
6. Unstage the staged wait that has prepare_to_wait_event()'ed *and* yet to get to __schedule(), if we encounter __schedule() in-between for another sleep, which is possible if e.g. a mutex_lock() exists in 'condition' of ___wait_event().
7. Turn on CONFIG_PROVE_LOCKING when CONFIG_DEPT is on, to rely on the hardirq and softirq entrance tracing to make Dept more portable for now.
Changes from v4:
1. Fix some bugs that produce false alarms.
2. Distinguish each syscall context from another *for arm64*.
3. Make it not warn it but just print it in case Dept ring buffer gets exhausted. (feedback from Hyeonggon)
4. Explicitely describe "EXPERIMENTAL" and "Dept might produce false positive reports" in Kconfig. (feedback from Ted)
Changes from v3:
1. Dept shouldn't create dependencies between different depths of a class that were indicated by *_lock_nested(). Dept normally doesn't but it does once another lock class comes in. So fixed it. (feedback from Hyeonggon)
2. Dept considered a wait as a real wait once getting to __schedule() even if it has been set to TASK_RUNNING by wake up sources in advance. Fixed it so that Dept doesn't consider the case as a real wait. (feedback from Jan Kara)
3. Stop tracking dependencies with a map once the event associated with the map has been handled. Dept will start to work with the map again, on the next sleep.
Changes from v2:
1. Disable Dept on bit_wait_table[] in sched/wait_bit.c reporting a lot of false positives, which is my fault. Wait/event for bit_wait_table[] should've been tagged in a higher layer for better work, which is a future work. (feedback from Jan Kara)
2. Disable Dept on crypto_larval's completion to prevent a false positive.
Changes from v1:
1. Fix coding style and typo. (feedback from Steven)
2. Distinguish each work context from another in workqueue.
3. Skip checking lock acquisition with nest_lock, which is about correct lock usage that should be checked by Lockdep.
Changes from RFC(v0):
1. Prevent adding a wait tag at prepare_to_wait() but __schedule(). (feedback from Linus and Matthew)
2. Use try version at lockdep_acquire_cpus_lock() annotation.
3. Distinguish each syscall context from another.
Byungchul Park (25):
llist: Move llist_{head,node} definition to types.h
dept: Implement Dept(Dependency Tracker)
dept: Add single event dependency tracker APIs
dept: Add lock dependency tracker APIs
dept: Tie to Lockdep and IRQ tracing
dept: Add proc knobs to show stats and dependency graph
dept: Apply sdt_might_sleep_{start,end}() to
wait_for_completion()/complete()
dept: Apply sdt_might_sleep_{start,end}() to PG_{locked,writeback}
wait
dept: Apply sdt_might_sleep_{start,end}() to swait
dept: Apply sdt_might_sleep_{start,end}() to waitqueue wait
dept: Apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait
dept: Distinguish each syscall context from another
dept: Distinguish each work from another
dept: Add a mechanism to refill the internal memory pools on running
out
locking/lockdep, cpu/hotplus: Use a weaker annotation in AP thread
dept: Apply sdt_might_sleep_{start,end}() to dma fence wait
dept: Track timeout waits separately with a new Kconfig
dept: Apply timeout consideration to wait_for_completion()/complete()
dept: Apply timeout consideration to swait
dept: Apply timeout consideration to waitqueue wait
dept: Apply timeout consideration to hashed-waitqueue wait
dept: Apply timeout consideration to dma fence wait
dept: Record the latest one out of consecutive waits of the same class
dept: Make Dept able to work with an external wgen
dept: Track the potential waits of PG_{locked,writeback}
arch/arm64/kernel/syscall.c | 2 + arch/x86/entry/common.c | 4 + drivers/dma-buf/dma-fence.c | 5 + include/linux/completion.h | 30 +- include/linux/dept.h | 614 +++++++ include/linux/dept_ldt.h | 77 + include/linux/dept_sdt.h | 66 + include/linux/hardirq.h | 3 + include/linux/irqflags.h | 22 +- include/linux/llist.h | 8 - include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 +- include/linux/lockdep_types.h | 3 + include/linux/mm_types.h | 3 + include/linux/mutex.h | 1 + include/linux/page-flags.h | 112 +- include/linux/pagemap.h | 7 +- include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/sched.h | 3 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- include/linux/swait.h | 3 + include/linux/types.h | 8 + include/linux/wait.h | 3 + include/linux/wait_bit.h | 3 + init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/cpu.c | 2 +- kernel/dependency/Makefile | 4 + kernel/dependency/dept.c | 3167 +++++++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_internal.h | 26 + kernel/dependency/dept_object.h | 13 + kernel/dependency/dept_proc.c | 93 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/locking/lockdep.c | 23 + kernel/module/main.c | 2 + kernel/sched/completion.c | 2 +- kernel/sched/core.c | 9 + kernel/workqueue.c | 3 + lib/Kconfig.debug | 37 + lib/locking-selftest.c | 2 + mm/filemap.c | 18 + mm/page_alloc.c | 3 + 50 files changed, 4459 insertions(+), 55 deletions(-) create mode 100644 include/linux/dept.h create mode 100644 include/linux/dept_ldt.h create mode 100644 include/linux/dept_sdt.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_object.h create mode 100644 kernel/dependency/dept_proc.c
Signature · 1 line
1.9.1
PATCH v9 1/25 llist: Move llist_{head,node} definition to types.h
llist_head and llist_node can be used by very primitives. For example, Dept for tracking dependency uses llist things in its header. To avoid header dependency, move those to types.h.
--- include/linux/llist.h | 8 -------- include/linux/types.h | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-)
include/linux/llist.h +0 -8
@@ -53,14 +53,6 @@#include <linux/stddef.h> #include <linux/types.h> -struct llist_head { -struct llist_node *first; -}; - -struct llist_node { -struct llist_node *next; -}; -#define LLIST_HEAD_INIT (name) { NULL }#define LLIST_HEAD (name) struct llist_head name = LLIST_HEAD_INIT(name)
include/linux/types.h +8 -0
@@ -187,6 +187,14 @@ struct hlist_node {struct hlist_node *next, **pprev; }; +struct llist_head { +struct llist_node *first; +}; + +struct llist_node { +struct llist_node *next; +}; +struct ustat {__kernel_daddr_t f_tfree;#ifdef CONFIG_ARCH_32BIT_USTAT_F_TINODE
Signature · 1 line
1.9.1
PATCH v9 2/25 dept: Implement Dept(Dependency Tracker)
In reply to Byungchul ParkCURRENT STATUS
--------------
Lockdep tracks acquisition order of locks in order to detect deadlock, and IRQ and IRQ enable/disable state as well to take accident acquisitions into account.
Lockdep should be turned off once it detects and reports a deadlock since the data structure and algorithm are not reusable after detection because of the complex design.
PROBLEM
-------
*Waits* and their *events* that never reach eventually cause deadlock. However, Lockdep is only interested in lock acquisition order, forcing to emulate lock acqusition even for just waits and events that have nothing to do with real lock.
Even worse, no one likes Lockdep's false positive detection because that prevents further one that might be more valuable. That's why all the kernel developers are sensitive to Lockdep's false positive.
Besides those, by tracking acquisition order, it cannot correctly deal with read lock and cross-event e.g. wait_for_completion()/complete() for deadlock detection. Lockdep is no longer a good tool for that purpose.
SOLUTION
--------
Again, *waits* and their *events* that never reach eventually cause deadlock. The new solution, Dept(DEPendency Tracker), focuses on waits and events themselves. Dept tracks waits and events and report it if any event would be never reachable.
Dept does:
. Works with read lock in the right way.
. Works with any wait and event e.i. cross-event. . Continue to work even after reporting multiple times. . Provides simple and intuitive APIs.
. Does exactly what dependency checker should do.
Q & A
-----
Q. Is this the first try ever to address the problem?
A. No. Cross-release feature (b09be676e0ff2 locking/lockdep: Implement
the 'crossrelease' feature) addressed it 2 years ago that was a Lockdep extension and merged but reverted shortly because:
Cross-release started to report valuable hidden problems but started to give report false positive reports as well. For sure, no one likes Lockdep's false positive reports since it makes Lockdep stop, preventing reporting further real problems.
Q. Why not Dept was developed as an extension of Lockdep? A. Lockdep definitely includes all the efforts great developers have
made for a long time so as to be quite stable enough. But I had to design and implement newly because of the following:
1) Lockdep was designed to track lock acquisition order. The APIs and implementation do not fit on wait-event model.
2) Lockdep is turned off on detection including false positive. Which is terrible and prevents developing any extension for stronger detection.
Q. Do you intend to totally replace Lockdep?
A. No. Lockdep also checks if lock usage is correct. Of course, the
dependency check routine should be replaced but the other functions should be still there.
Q. Do you mean the dependency check routine should be replaced right
away?
A. No. I admit Lockdep is stable enough thanks to great efforts kernel
developers have made. Lockdep and Dept, both should be in the kernel until Dept gets considered stable.
Q. Stronger detection capability would give more false positive report.
Which was a big problem when cross-release was introduced. Is it ok with Dept?
A. It's ok. Dept allows multiple reporting thanks to simple and quite
generalized design. Of course, false positive reports should be fixed anyway but it's no longer as a critical problem as it was.
--- include/linux/dept.h | 577 ++++++++ include/linux/hardirq.h | 3 + include/linux/sched.h | 3 + init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/dependency/Makefile | 3 + kernel/dependency/dept.c | 3009 +++++++++++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_object.h | 13 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/module/main.c | 2 + kernel/sched/core.c | 9 + lib/Kconfig.debug | 27 + lib/locking-selftest.c | 2 + 16 files changed, 3666 insertions(+) create mode 100644 include/linux/dept.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_object.h
include/linux/dept.h new file +577 -0
@@ -0,0 +1,577 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* DEPT(DEPendency Tracker) - runtime dependency tracker +* +* Started by Byungchul Park <[email protected]>: +* +* Copyright (c) 2020 LG Electronics, Inc., Byungchul Park +*/ + +#ifndef __LINUX_DEPT_H +#define __LINUX_DEPT_H + +#ifdef CONFIG_DEPT + +#include <linux/types.h> + +struct task_struct ; + +#define DEPT_MAX_STACK_ENTRY 16 +#define DEPT_MAX_WAIT_HIST 64 +#define DEPT_MAX_ECXT_HELD 48 + +#define DEPT_MAX_SUBCLASSES 16 +#define DEPT_MAX_SUBCLASSES_EVT 2 +#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) +#define DEPT_MAX_SUBCLASSES_CACHE 2 + +#define DEPT_SIRQ 0 +#define DEPT_HIRQ 1 +#define DEPT_IRQS_NR 2 +#define DEPT_SIRQF (1UL << DEPT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_HIRQ) + +struct dept_ecxt ; +struct dept_iecxt { +struct dept_ecxt *ecxt; +int enirq; +/* +* for preventing to add a new ecxt +*/ +bool staled; +}; + +struct dept_wait ; +struct dept_iwait { +struct dept_wait *wait; +int irq; +/* +* for preventing to add a new wait +*/ +bool staled; +bool touched; +}; + +struct dept_class { +union { +struct llist_node pool_node; +struct { +/* +* reference counter for object management +*/ +atomic_t ref; + +/* +* unique information about the class +*/ +const char *name; +unsigned long key; +int sub_id; + +/* +* for BFS +*/ +unsigned int bfs_gen; +int bfs_dist; +struct dept_class *bfs_parent; + +/* +* for hashing this object +*/ +struct hlist_node hash_node; + +/* +* for linking all classes +*/ +struct list_head all_node; + +/* +* for associating its dependencies +*/ +struct list_head dep_head; +struct list_head dep_rev_head; + +/* +* for tracking IRQ dependencies +*/ +struct dept_iecxt iecxt[DEPT_IRQS_NR ]; +struct dept_iwait iwait[DEPT_IRQS_NR ]; + +/* +* classified by a map embedded in task_struct, +* not an explicit map +*/ +bool sched_map; + }; + }; +}; + +struct dept_key { +union { +/* +* Each byte-wise address will be used as its key. +*/ +char base[DEPT_MAX_SUBCLASSES ]; + +/* +* for caching the main class pointer +*/ +struct dept_class *classes[DEPT_MAX_SUBCLASSES_CACHE ]; + }; +}; + +struct dept_map { +const char *name; +struct dept_key *keys; + +/* +* subclass that can be set from user +*/ +int sub_u; + +/* +* It's local copy for fast access to the associated classes. +* Also used for dept_key for static maps. +*/ +struct dept_key map_key; + +/* +* wait timestamp associated to this map +*/ +unsigned int wgen; + +/* +* whether this map should be going to be checked or not +*/ +bool nocheck; +}; + +#define DEPT_MAP_INITIALIZER (n, k) \ +{ \ + .name = #n, \ + .keys = (struct dept_key *)(k), \ + .sub_u = 0, \ + .map_key = { .classes = { NULL, } }, \ + .wgen = 0U, \ + .nocheck = false, \ +} + +struct dept_stack { +union { +struct llist_node pool_node; +struct { +/* +* reference counter for object management +*/ +atomic_t ref; + +/* +* backtrace entries +*/ +unsigned long raw[DEPT_MAX_STACK_ENTRY ]; +int nr; + }; + }; +}; + +struct dept_ecxt { +union { +struct llist_node pool_node; +struct { +/* +* reference counter for object management +*/ +atomic_t ref; + +/* +* function that entered to this ecxt +*/ +const char *ecxt_fn; + +/* +* event function +*/ +const char *event_fn; + +/* +* associated class +*/ +struct dept_class *class; + +/* +* flag indicating which IRQ has been +* enabled within the event context +*/ +unsigned long enirqf; + +/* +* where the IRQ-enabled happened +*/ +unsigned long enirq_ip[DEPT_IRQS_NR ]; +struct dept_stack *enirq_stack[DEPT_IRQS_NR ]; + +/* +* where the event context started +*/ +unsigned long ecxt_ip; +struct dept_stack *ecxt_stack; + +/* +* where the event triggered +*/ +unsigned long event_ip; +struct dept_stack *event_stack; + }; + }; +}; + +struct dept_wait { +union { +struct llist_node pool_node; +struct { +/* +* reference counter for object management +*/ +atomic_t ref; + +/* +* function causing this wait +*/ +const char *wait_fn; + +/* +* the associated class +*/ +struct dept_class *class; + +/* +* which IRQ the wait was placed in +*/ +unsigned long irqf; + +/* +* where the IRQ wait happened +*/ +unsigned long irq_ip[DEPT_IRQS_NR ]; +struct dept_stack *irq_stack[DEPT_IRQS_NR ]; + +/* +* where the wait happened +*/ +unsigned long wait_ip; +struct dept_stack *wait_stack; + +/* +* whether this wait is for commit in scheduler +*/ +bool sched_sleep; + }; + }; +}; + +struct dept_dep { +union { +struct llist_node pool_node; +struct { +/* +* reference counter for object management +*/ +atomic_t ref; + +/* +* key data of dependency +*/ +struct dept_ecxt *ecxt; +struct dept_wait *wait; + +/* +* This object can be referred without dept_lock +* held but with IRQ disabled, e.g. for hash +* lookup. So deferred deletion is needed. +*/ +struct rcu_head rh; + +/* +* for BFS +*/ +struct list_head bfs_node; + +/* +* for hashing this object +*/ +struct hlist_node hash_node; + +/* +* for linking to a class object +*/ +struct list_head dep_node; +struct list_head dep_rev_node; + }; + }; +}; + +struct dept_hash { +/* +* hash table +*/ +struct hlist_head *table; + +/* +* size of the table e.i. 2^bits +*/ +int bits; +}; + +struct dept_pool { +const char *name; + +/* +* object size +*/ +size_t obj_sz; + +/* +* the number of the static array +*/ +atomic_t obj_nr; + +/* +* offset of ->pool_node +*/ +size_t node_off; + +/* +* pointer to the pool +*/ +void *spool; +struct llist_head boot_pool; +struct llist_head __percpu *lpool; +}; + +struct dept_ecxt_held { +/* +* associated event context +*/ +struct dept_ecxt *ecxt; + +/* +* unique key for this dept_ecxt_held +*/ +struct dept_map *map; + +/* +* class of the ecxt of this dept_ecxt_held +*/ +struct dept_class *class; + +/* +* the wgen when the event context started +*/ +unsigned int wgen; + +/* +* subclass that only works in the local context +*/ +int sub_l; +}; + +struct dept_wait_hist { +/* +* associated wait +*/ +struct dept_wait *wait; + +/* +* unique id of all waits system-wise until wrapped +*/ +unsigned int wgen; + +/* +* local context id to identify IRQ context +*/ +unsigned int ctxt_id; +}; + +struct dept_task { +/* +* all event contexts that have entered and before exiting +*/ +struct dept_ecxt_held ecxt_held[DEPT_MAX_ECXT_HELD ]; +int ecxt_held_pos; + +/* +* ring buffer holding all waits that have happened +*/ +struct dept_wait_hist wait_hist[DEPT_MAX_WAIT_HIST ]; +int wait_hist_pos; + +/* +* sequential id to identify each IRQ context +*/ +unsigned int irq_id[DEPT_IRQS_NR ]; + +/* +* for tracking IRQ-enabled points with cross-event +*/ +unsigned int wgen_enirq[DEPT_IRQS_NR ]; + +/* +* for keeping up-to-date IRQ-enabled points +*/ +unsigned long enirq_ip[DEPT_IRQS_NR ]; + +/* +* current effective IRQ-enabled flag +*/ +unsigned long eff_enirqf; + +/* +* for reserving a current stack instance at each operation +*/ +struct dept_stack *stack; + +/* +* for preventing recursive call into DEPT engine +*/ +int recursive; + +/* +* for staging data to commit a wait +*/ +struct dept_map stage_m; +bool stage_sched_map; +const char *stage_w_fn; +unsigned long stage_ip; + +/* +* the number of missing ecxts +*/ +int missing_ecxt; + +/* +* for tracking IRQ-enable state +*/ +bool hardirqs_enabled; +bool softirqs_enabled; + +/* +* whether the current is on do_exit() +*/ +bool task_exit; + +/* +* whether the current is running __schedule() +*/ +bool in_sched; +}; + +#define DEPT_TASK_INITIALIZER (t) \ +{ \ + .wait_hist = { { .wait = NULL, } }, \ + .ecxt_held_pos = 0, \ + .wait_hist_pos = 0, \ + .irq_id = { 0U }, \ + .wgen_enirq = { 0U }, \ + .enirq_ip = { 0UL }, \ + .eff_enirqf = 0UL, \ + .stack = NULL, \ + .recursive = 0, \ + .stage_m = DEPT_MAP_INITIALIZER((t)->stage_m, NULL), \ + .stage_sched_map = false, \ + .stage_w_fn = NULL, \ + .stage_ip = 0UL, \ + .missing_ecxt = 0, \ + .hardirqs_enabled = false, \ + .softirqs_enabled = false, \ + .task_exit = false, \ + .in_sched = false, \ +} + +extern void dept_on (void ); +extern void dept_off (void ); +extern void dept_init (void ); +extern void dept_task_init (struct task_struct *t); +extern void dept_task_exit (struct task_struct *t); +extern void dept_free_range (void *start,unsigned int sz); +extern void dept_map_init (struct dept_map *m,struct dept_key *k,int sub_u,const char *n); +extern void dept_map_reinit (struct dept_map *m,struct dept_key *k,int sub_u,const char *n); +extern void dept_map_copy (struct dept_map *to,struct dept_map *from); + +extern void dept_wait (struct dept_map *m,unsigned long w_f,unsigned long ip,const char *w_fn,int sub_l); +extern void dept_stage_wait (struct dept_map *m,struct dept_key *k,unsigned long ip,const char *w_fn); +extern void dept_request_event_wait_commit (void ); +extern void dept_clean_stage (void ); +extern void dept_stage_event (struct task_struct *t,unsigned long ip); +extern void dept_ecxt_enter (struct dept_map *m,unsigned long e_f,unsigned long ip,const char *c_fn,const char *e_fn,int sub_l); +extern bool dept_ecxt_holding (struct dept_map *m,unsigned long e_f); +extern void dept_request_event (struct dept_map *m); +extern void dept_event (struct dept_map *m,unsigned long e_f,unsigned long ip,const char *e_fn); +extern void dept_ecxt_exit (struct dept_map *m,unsigned long e_f,unsigned long ip); +extern void dept_sched_enter (void ); +extern void dept_sched_exit (void ); + +static inline void dept_ecxt_enter_nokeep (struct dept_map *m) +{ +dept_ecxt_enter (m,0UL ,0UL ,NULL ,NULL ,0 ); +} + +/* +* for users who want to manage external keys +*/ +extern void dept_key_init (struct dept_key *k); +extern void dept_key_destroy (struct dept_key *k); +extern void dept_map_ecxt_modify (struct dept_map *m,unsigned long e_f,struct dept_key *new_k,unsigned long new_e_f,unsigned long new_ip,const char *new_c_fn,const char *new_e_fn,int new_sub_l); + +extern void dept_softirq_enter (void ); +extern void dept_hardirq_enter (void ); +extern void dept_softirqs_on_ip (unsigned long ip); +extern void dept_hardirqs_on (void ); +extern void dept_hardirqs_on_ip (unsigned long ip); +extern void dept_softirqs_off_ip (unsigned long ip); +extern void dept_hardirqs_off (void ); +extern void dept_hardirqs_off_ip (unsigned long ip); +#else /* !CONFIG_DEPT */ +struct dept_key { }; +struct dept_map { }; +struct dept_task { }; + +#define DEPT_MAP_INITIALIZER (n, k) { } +#define DEPT_TASK_INITIALIZER (t) { } + +#define dept_on () do { } while (0) +#define dept_off () do { } while (0) +#define dept_init () do { } while (0) +#define dept_task_init (t) do { } while (0) +#define dept_task_exit (t) do { } while (0) +#define dept_free_range (s, sz) do { } while (0) +#define dept_map_init (m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_reinit (m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_copy (t, f) do { } while (0) + +#define dept_wait (m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) +#define dept_stage_wait (m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_request_event_wait_commit () do { } while (0) +#define dept_clean_stage () do { } while (0) +#define dept_stage_event (t, ip) do { } while (0) +#define dept_ecxt_enter (m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) +#define dept_ecxt_holding (m, e_f) false +#define dept_request_event (m) do { } while (0) +#define dept_event (m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_ecxt_exit (m, e_f, ip) do { } while (0) +#define dept_sched_enter () do { } while (0) +#define dept_sched_exit () do { } while (0) +#define dept_ecxt_enter_nokeep (m) do { } while (0) +#define dept_key_init (k) do { (void)(k); } while (0) +#define dept_key_destroy (k) do { (void)(k); } while (0) +#define dept_map_ecxt_modify (m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_sl) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) + +#define dept_softirq_enter () do { } while (0) +#define dept_hardirq_enter () do { } while (0) +#define dept_softirqs_on_ip (ip) do { } while (0) +#define dept_hardirqs_on () do { } while (0) +#define dept_hardirqs_on_ip (ip) do { } while (0) +#define dept_softirqs_off_ip (ip) do { } while (0) +#define dept_hardirqs_off () do { } while (0) +#define dept_hardirqs_off_ip (ip) do { } while (0) +#endif +#endif /* __LINUX_DEPT_H */
include/linux/hardirq.h +3 -0
@@ -5,6 +5,7 @@#include <linux/context_tracking_state.h> #include <linux/preempt.h> #include <linux/lockdep.h> +#include <linux/dept.h> #include <linux/ftrace_irq.h> #include <linux/sched.h> #include <linux/vtime.h>
@@ -106,6 +107,7 @@ static __always_inline void rcu_irq_enter_check_tick(void)*/ #define __nmi_enter () \ do { \ + dept_off(); \ lockdep_off(); \ arch_nmi_enter(); \ BUG_ON(in_nmi() == NMI_MASK); \
@@ -128,6 +130,7 @@ static __always_inline void rcu_irq_enter_check_tick(void)__preempt_count_sub (NMI_OFFSET +HARDIRQ_OFFSET ); \arch_nmi_exit (); \lockdep_on (); \ +dept_on (); \ }while (0 ) #define nmi_exit() \
include/linux/sched.h +3 -0
@@ -37,6 +37,7 @@#include <linux/kcsan.h> #include <linux/rv.h> #include <asm/kmap_size.h> +#include <linux/dept.h> /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context ;
@@ -1168,6 +1169,8 @@ struct task_struct {struct held_lock held_locks[MAX_LOCK_DEPTH ];#endif +struct dept_task dept_task; +#if defined(CONFIG_UBSAN ) && !defined(CONFIG_UBSAN_TRAP )unsigned int in_ubsan;#endif
init/init_task.c +2 -0
@@ -12,6 +12,7 @@#include <linux/audit.h> #include <linux/numa.h> #include <linux/scs.h> +#include <linux/dept.h> #include <linux/uaccess.h>
@@ -194,6 +195,7 @@ struct task_struct init_task .curr_chain_key = INITIAL_CHAIN_KEY, .lockdep_recursion = 0, #endif + .dept_task =DEPT_TASK_INITIALIZER (init_task), #ifdef CONFIG_FUNCTION_GRAPH_TRACER .ret_stack =NULL , .tracing_graph_pause = ATOMIC_INIT(0),
init/main.c +2 -0
@@ -66,6 +66,7 @@#include <linux/debug_locks.h> #include <linux/debugobjects.h> #include <linux/lockdep.h> +#include <linux/dept.h> #include <linux/kmemleak.h> #include <linux/padata.h> #include <linux/pid_namespace.h>
@@ -1080,6 +1081,7 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void) panic_param);lockdep_init (); +dept_init ();/* * Need to run this when irqs are enabled, because it wants
kernel/Makefile +1 -0
@@ -51,6 +51,7 @@ obj-y += livepatch/ obj-y += dma/ obj-y += entry/ obj-$(CONFIG_MODULES) += module/ +obj-y += dependency/ obj-$(CONFIG_KCMP) += kcmp.o obj-$(CONFIG_FREEZER) += freezer.o
kernel/dependency/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DEPT) += dept.o
kernel/dependency/dept.c new file +3009 -0
2,349 lines not on this page · raw message
kernel/dependency/dept_hash.h new file +10 -0
Lines of this file are not shown. View the raw message.
kernel/dependency/dept_object.h new file +13 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
1.9.1
Part of the diff is not shown here. View the raw message.
PATCH v9 3/25 dept: Add single event dependency tracker APIs
In reply to Byungchul ParkWrapped the base APIs for easier annotation on wait and event. Start with supporting waiters on each single event. More general support for multiple events is a future work. Do more when the need arises.
How to annotate (the simplest way):
1. Initaialize a map for the interesting wait.
/*
* Recommand to place along with the wait instance. */
struct dept_map my_wait;
/*
* Recommand to place in the initialization code. */
sdt_map_init(&my_wait);
2. Place the following at the wait code.
sdt_wait(&my_wait);
3. Place the following at the event code.
sdt_event(&my_wait);
That's it!
--- include/linux/dept_sdt.h | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 include/linux/dept_sdt.h
include/linux/dept_sdt.h new file +62 -0
@@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Single-event Dependency Tracker +* +* Started by Byungchul Park <[email protected]>: +* +* Copyright (c) 2020 LG Electronics, Inc., Byungchul Park +*/ + +#ifndef __LINUX_DEPT_SDT_H +#define __LINUX_DEPT_SDT_H + +#include <linux/kernel.h> +#include <linux/dept.h> + +#ifdef CONFIG_DEPT +#define sdt_map_init (m) \ + do { \ + static struct dept_key __key; \ + dept_map_init(m, &__key, 0, #m); \ + } while (0) + +#define sdt_map_init_key (m, k) dept_map_init(m, k, 0, #m) + +#define sdt_wait (m) \ + do { \ + dept_request_event(m); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + } while (0) + +/* +* sdt_might_sleep() and its family will be committed in __schedule() +* when it actually gets to __schedule(). Both dept_request_event() and +* dept_wait() will be performed on the commit. +*/ + +/* +* Use the code location as the class key if an explicit map is not used. +*/ +#define sdt_might_sleep_start (m) \ + do { \ + struct dept_map *__m = m; \ + static struct dept_key __key; \ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + } while (0) + +#define sdt_might_sleep_end () dept_clean_stage() + +#define sdt_ecxt_enter (m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) +#define sdt_event (m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_ecxt_exit (m) dept_ecxt_exit(m, 1UL, _THIS_IP_) +#else /* !CONFIG_DEPT */ +#define sdt_map_init (m) do { } while (0) +#define sdt_map_init_key (m, k) do { (void)(k); } while (0) +#define sdt_wait (m) do { } while (0) +#define sdt_might_sleep_start (m) do { } while (0) +#define sdt_might_sleep_end () do { } while (0) +#define sdt_ecxt_enter (m) do { } while (0) +#define sdt_event (m) do { } while (0) +#define sdt_ecxt_exit (m) do { } while (0) +#endif +#endif /* __LINUX_DEPT_SDT_H */
Signature · 1 line
1.9.1
PATCH v9 4/25 dept: Add lock dependency tracker APIs
In reply to Byungchul ParkWrapped the base APIs for easier annotation on typical lock.
--- include/linux/dept_ldt.h | 77 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 include/linux/dept_ldt.h
include/linux/dept_ldt.h new file +77 -0
@@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Lock Dependency Tracker +* +* Started by Byungchul Park <[email protected]>: +* +* Copyright (c) 2020 LG Electronics, Inc., Byungchul Park +*/ + +#ifndef __LINUX_DEPT_LDT_H +#define __LINUX_DEPT_LDT_H + +#include <linux/dept.h> + +#ifdef CONFIG_DEPT +#define LDT_EVT_L 1UL +#define LDT_EVT_R 2UL +#define LDT_EVT_W 1UL +#define LDT_EVT_RW (LDT_EVT_R | LDT_EVT_W) +#define LDT_EVT_ALL (LDT_EVT_L | LDT_EVT_RW) + +#define ldt_init (m, k, su, n) dept_map_init(m, k, su, n) +#define ldt_lock (m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ + } \ + } while (0) + +#define ldt_rlock (m, sl, t, n, i, q) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ + else { \ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ + } \ + } while (0) + +#define ldt_wlock (m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ + } \ + } while (0) + +#define ldt_unlock (m, i) dept_ecxt_exit(m, LDT_EVT_ALL, i) + +#define ldt_downgrade (m, i) \ + do { \ + if (dept_ecxt_holding(m, LDT_EVT_W)) \ + dept_map_ecxt_modify(m, LDT_EVT_W, NULL, LDT_EVT_R, i, "downgrade", "read_unlock", -1);\ + } while (0) + +#define ldt_set_class (m, n, k, sl, i) dept_map_ecxt_modify(m, LDT_EVT_ALL, k, 0UL, i, "lock_set_class", "(any)unlock", sl) +#else /* !CONFIG_DEPT */ +#define ldt_init (m, k, su, n) do { (void)(k); } while (0) +#define ldt_lock (m, sl, t, n, i) do { } while (0) +#define ldt_rlock (m, sl, t, n, i, q) do { } while (0) +#define ldt_wlock (m, sl, t, n, i) do { } while (0) +#define ldt_unlock (m, i) do { } while (0) +#define ldt_downgrade (m, i) do { } while (0) +#define ldt_set_class (m, n, k, sl, i) do { } while (0) +#endif +#endif /* __LINUX_DEPT_LDT_H */
Signature · 1 line
1.9.1
PATCH v9 5/25 dept: Tie to Lockdep and IRQ tracing
In reply to Byungchul ParkYes. How to place Dept in here looks so ugly. But it's inevitable as long as relying on Lockdep. The way should be enhanced gradually.
1. Basically relies on Lockdep to track typical locks and IRQ things.
2. Dept fails to recognize IRQ situation so it generates false alarms when raw_local_irq_*() APIs are used. So made it track those too.
3. Lockdep doesn't track the outmost {hard,soft}irq entracnes but Dept makes use of it. So made it track those too.
--- include/linux/irqflags.h | 22 ++++++-- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++++++++-------- include/linux/lockdep_types.h | 3 ++ include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 ++ include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 4 +- kernel/locking/lockdep.c | 23 ++++++++ 14 files changed, 139 insertions(+), 29 deletions(-)
include/linux/irqflags.h +19 -3
@@ -13,6 +13,7 @@#define _LINUX_TRACE_IRQFLAGS_H#include <linux/typecheck.h> +#include <linux/dept.h> #include <asm/irqflags.h> #include <asm/percpu.h>
@@ -60,8 +61,10 @@ struct irqtrace_events {# define lockdep_softirqs_enabled (p) ((p)->softirqs_enabled)# define lockdep_hardirq_enter () \ do { \ - if (__this_cpu_inc_return(hardirq_context) == 1)\ + if (__this_cpu_inc_return(hardirq_context) == 1) { \ current->hardirq_threaded = 0; \ + dept_hardirq_enter(); \ + } \ } while (0)# define lockdep_hardirq_threaded () \ do { \
@@ -136,6 +139,8 @@ struct irqtrace_events {# define lockdep_softirq_enter () \ do { \ current->softirq_context++; \ + if (current->softirq_context == 1) \ + dept_softirq_enter(); \ } while (0)# define lockdep_softirq_exit () \ do { \
@@ -170,17 +175,28 @@ struct irqtrace_events {/* * Wrap the arch provided IRQ routines to provide appropriate checks. */ -#define raw_local_irq_disable () arch_local_irq_disable() -#define raw_local_irq_enable () arch_local_irq_enable() +#define raw_local_irq_disable () \ + do { \ + arch_local_irq_disable(); \ + dept_hardirqs_off(); \ + } while (0) +#define raw_local_irq_enable () \ + do { \ + dept_hardirqs_on(); \ + arch_local_irq_enable(); \ + } while (0)#define raw_local_irq_save (flags) \ do { \ typecheck(unsigned long, flags); \ flags = arch_local_irq_save(); \ + dept_hardirqs_off(); \ } while (0)#define raw_local_irq_restore (flags) \ do { \ typecheck(unsigned long, flags); \ raw_check_bogus_irq_restore(); \ + if (!arch_irqs_disabled_flags(flags)) \ + dept_hardirqs_on(); \ arch_local_irq_restore(flags); \ } while (0)#define raw_local_save_flags (flags) \
include/linux/local_lock_internal.h +1 -0
@@ -21,6 +21,7 @@ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner = NULL,
include/linux/lockdep.h +81 -21
133 lines not on this page · raw message
include/linux/lockdep_types.h +3 -0
@@ -11,6 +11,7 @@#define __LINUX_LOCKDEP_TYPES_H#include <linux/types.h> +#include <linux/dept.h> #define MAX_LOCKDEP_SUBCLASSES 8UL
@@ -76,6 +77,7 @@ struct lock_class_key {struct hlist_node hash_entry;struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES ]; }; +struct dept_key dkey; };extern struct lock_class_key __lockdep_no_validate__;
@@ -185,6 +187,7 @@ struct lockdep_map {int cpu;unsigned long ip;#endif +struct dept_map dmap; };struct pin_cookie {unsigned int val; };
include/linux/mutex.h +1 -0
@@ -25,6 +25,7 @@ , .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }#else # define __DEP_MAP_MUTEX_INITIALIZER (lockname)
include/linux/percpu-rwsem.h +1 -1
@@ -21,7 +21,7 @@ struct percpu_rw_semaphore { };#ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT (lockname) .dep_map = { .name = #lockname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT (lockname) .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) },#else #define __PERCPU_RWSEM_DEP_MAP_INIT (lockname)#endif
include/linux/rtmutex.h +1 -0
@@ -81,6 +81,7 @@ static inline void rt_mutex_debug_task_free(struct task_struct *tsk) { } .dep_map = { \ .name = #mutexname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(mutexname, NULL),\ }#else #define __DEP_MAP_RT_MUTEX_INITIALIZER (mutexname)
include/linux/rwlock_types.h +1 -0
@@ -10,6 +10,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL), \ }#else # define RW_DEP_MAP_INIT (lockname)
include/linux/rwsem.h +1 -0
@@ -21,6 +21,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ },#else # define __RWSEM_DEP_MAP_INIT (lockname)
include/linux/seqlock.h +1 -1
@@ -81,7 +81,7 @@ static inline void __seqcount_init(seqcount_t *s, const char *name,#ifdef CONFIG_DEBUG_LOCK_ALLOC # define SEQCOUNT_DEP_MAP_INIT (lockname) \ - .dep_map = { .name = #lockname } + .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) }/** * seqcount_init() - runtime initializer for seqcount_t
include/linux/spinlock_types_raw.h +3 -0
@@ -31,11 +31,13 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SPIN, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }# define SPIN_DEP_MAP_INIT (lockname) \ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }# define LOCAL_SPIN_DEP_MAP_INIT (lockname) \
@@ -43,6 +45,7 @@ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }#else # define RAW_SPIN_DEP_MAP_INIT (lockname)
include/linux/srcu.h +1 -1
@@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const char *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) -#define __SRCU_DEP_MAP_INIT (srcu_name) .dep_map = { .name = #srcu_name }, +#define __SRCU_DEP_MAP_INIT (srcu_name) .dep_map = { .name = #srcu_name, .dmap = DEPT_MAP_INITIALIZER(srcu_name, NULL) },#else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ int init_srcu_struct (struct srcu_struct *ssp);
kernel/dependency/dept.c +2 -2
@@ -245,10 +245,10 @@ static inline bool dept_working(void)* Even k == NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__;static inline bool valid_key (struct dept_key *k) { -return &__dept_no_validate__ != k; +return &__lockdep_no_validate__.dkey != k; }/*
kernel/locking/lockdep.c +23 -0
@@ -1220,6 +1220,8 @@ void lockdep_register_key(struct lock_class_key *key)struct lock_class_key *k;unsigned long flags; +dept_key_init (&key->dkey); +if (WARN_ON_ONCE (static_obj (key)))return ; hash_head =keyhashentry (key);
@@ -4327,6 +4329,8 @@ void noinstr lockdep_hardirqs_on(unsigned long ip) {struct irqtrace_events *trace = ¤t->irqtrace; +dept_hardirqs_on_ip (ip); +if (unlikely (!debug_locks))return ;
@@ -4392,6 +4396,8 @@ void noinstr lockdep_hardirqs_on(unsigned long ip)*/ void noinstr lockdep_hardirqs_off(unsigned long ip) { +dept_hardirqs_off_ip (ip); +if (unlikely (!debug_locks))return ;
@@ -4436,6 +4442,8 @@ void lockdep_softirqs_on(unsigned long ip) {struct irqtrace_events *trace = ¤t->irqtrace; +dept_softirqs_on_ip (ip); +if (unlikely (!lockdep_enabled ()))return ;
@@ -4474,6 +4482,9 @@ void lockdep_softirqs_on(unsigned long ip)*/ void lockdep_softirqs_off (unsigned long ip) { + +dept_softirqs_off_ip (ip); +if (unlikely (!lockdep_enabled ()))return ;
@@ -4806,6 +4817,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, {int i; +ldt_init (&lock->dmap, &key->dkey, subclass, name); +for (i =0 ; i <NR_LOCKDEP_CACHING_CLASSES ; i++) lock->class_cache[i] =NULL ;
@@ -5544,6 +5557,12 @@ void lock_set_class(struct lockdep_map *lock, const char *name, {unsigned long flags; +/* +* dept_map_(re)init() might be called twice redundantly. But +* there's no choice as long as Dept relies on Lockdep. +*/ +ldt_set_class (&lock->dmap, name, &key->dkey, subclass, ip); +if (unlikely (!lockdep_enabled ()))return ;
@@ -5561,6 +5580,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigned long ip) {unsigned long flags; +ldt_downgrade (&lock->dmap, ip); +if (unlikely (!lockdep_enabled ()))return ;
@@ -6333,6 +6354,8 @@ void lockdep_unregister_key(struct lock_class_key *key)unsigned long flags;bool found = false; +dept_key_destroy (&key->dkey); +might_sleep ();if (WARN_ON_ONCE (static_obj (key)))
Signature · 1 line
1.9.1
kernel test robot
build botRe: [PATCH v9 05/25] dept: Tie to Lockdep and IRQ tracing
Thank you for the patch! Perhaps something to improve:
Hi Byungchul,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on tip/locking/core]
[also build test WARNING on tip/sched/core drm-misc/drm-misc-next linus/master v6.2-rc6 next-20230131] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Byungchul-Park/llist-Move-llist_-head-node-definition-to-types-h/20230131-164632 patch link: https://lore.kernel.org/r/1675154394-25598-6-git-send-email-max.byungchul.park%40gmail.com patch subject: [PATCH v9 05/25] dept: Tie to Lockdep and IRQ tracing config: arc-allyesconfig (https://download.01.org/0day-ci/archive/20230201/[email protected]/config) compiler: arceb-elf-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/ee619b0e7faff81e8b794e2adcdf9102d5bc83e8
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Byungchul-Park/llist-Move-llist_-head-node-definition-to-types-h/20230131-164632
git checkout ee619b0e7faff81e8b794e2adcdf9102d5bc83e8
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arc olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash drivers/iommu/ drivers/scsi/qla2xxx/
If you fix the issue, kindly add following tag where applicable | Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
In file included from include/linux/atomic.h:80,
from drivers/iommu/io-pgtable-arm.c:12:
drivers/iommu/io-pgtable-arm.c: In function 'arm_lpae_install_table':
include/linux/atomic/atomic-arch-fallback.h:60:32: error: implicit declaration of function 'arch_cmpxchg64'; did you mean 'arch_cmpxchg'? [-Werror=implicit-function-declaration]
60 | #define arch_cmpxchg64_relaxed arch_cmpxchg64
| ^~~~~~~~~~~~~~
include/linux/atomic/atomic-instrumented.h:1968:9: note: in expansion of macro 'arch_cmpxchg64_relaxed'
1968 | arch_cmpxchg64_relaxed(__ai_ptr, __VA_ARGS__); \
| ^~~~~~~~~~~~~~~~~~~~~~
drivers/iommu/io-pgtable-arm.c:330:15: note: in expansion of macro 'cmpxchg64_relaxed'
330 | old = cmpxchg64_relaxed(ptep, curr, new);
| ^~~~~~~~~~~~~~~~~
drivers/iommu/io-pgtable-arm.c: In function 'arm_lpae_do_selftests':
>> drivers/iommu/io-pgtable-arm.c:1296:1: warning: the frame size of 1088 bytes is larger than 1024 bytes [-Wframe-larger-than=]
1296 | }
| ^cc1: some warnings being treated as errors
vim +1296 drivers/iommu/io-pgtable-arm.c
fe4b991dcd84e0 Will Deacon 2014-11-17 1256
fe4b991dcd84e0 Will Deacon 2014-11-17 1257 static int __init arm_lpae_do_selftests(void)
fe4b991dcd84e0 Will Deacon 2014-11-17 1258 {
9062c1d0bedacf Christophe JAILLET 2019-09-09 1259 static const unsigned long pgsize[] __initconst = {
fe4b991dcd84e0 Will Deacon 2014-11-17 1260 SZ_4K | SZ_2M | SZ_1G,
fe4b991dcd84e0 Will Deacon 2014-11-17 1261 SZ_16K | SZ_32M,
fe4b991dcd84e0 Will Deacon 2014-11-17 1262 SZ_64K | SZ_512M,
fe4b991dcd84e0 Will Deacon 2014-11-17 1263 };
fe4b991dcd84e0 Will Deacon 2014-11-17 1264
9062c1d0bedacf Christophe JAILLET 2019-09-09 1265 static const unsigned int ias[] __initconst = {
fe4b991dcd84e0 Will Deacon 2014-11-17 1266 32, 36, 40, 42, 44, 48,
fe4b991dcd84e0 Will Deacon 2014-11-17 1267 };
fe4b991dcd84e0 Will Deacon 2014-11-17 1268
fe4b991dcd84e0 Will Deacon 2014-11-17 1269 int i, j, pass = 0, fail = 0;
ca25ec247aadbf Robin Murphy 2022-08-15 1270 struct device dev;
fe4b991dcd84e0 Will Deacon 2014-11-17 1271 struct io_pgtable_cfg cfg = {
fe4b991dcd84e0 Will Deacon 2014-11-17 1272 .tlb = &dummy_tlb_ops,
fe4b991dcd84e0 Will Deacon 2014-11-17 1273 .oas = 48,
4f41845b340783 Will Deacon 2019-06-25 1274 .coherent_walk = true,
ca25ec247aadbf Robin Murphy 2022-08-15 1275 .iommu_dev = &dev,
fe4b991dcd84e0 Will Deacon 2014-11-17 1276 };
fe4b991dcd84e0 Will Deacon 2014-11-17 1277
ca25ec247aadbf Robin Murphy 2022-08-15 1278 /* __arm_lpae_alloc_pages() merely needs dev_to_node() to work */
ca25ec247aadbf Robin Murphy 2022-08-15 1279 set_dev_node(&dev, NUMA_NO_NODE);
ca25ec247aadbf Robin Murphy 2022-08-15 1280
fe4b991dcd84e0 Will Deacon 2014-11-17 1281 for (i = 0; i < ARRAY_SIZE(pgsize); ++i) {
fe4b991dcd84e0 Will Deacon 2014-11-17 1282 for (j = 0; j < ARRAY_SIZE(ias); ++j) {
fe4b991dcd84e0 Will Deacon 2014-11-17 1283 cfg.pgsize_bitmap = pgsize[i];
fe4b991dcd84e0 Will Deacon 2014-11-17 1284 cfg.ias = ias[j];
fe4b991dcd84e0 Will Deacon 2014-11-17 1285 pr_info("selftest: pgsize_bitmap 0x%08lx, IAS %u\n",
fe4b991dcd84e0 Will Deacon 2014-11-17 1286 pgsize[i], ias[j]);
fe4b991dcd84e0 Will Deacon 2014-11-17 1287 if (arm_lpae_run_tests(&cfg))
fe4b991dcd84e0 Will Deacon 2014-11-17 1288 fail++;
fe4b991dcd84e0 Will Deacon 2014-11-17 1289 else
fe4b991dcd84e0 Will Deacon 2014-11-17 1290 pass++;
fe4b991dcd84e0 Will Deacon 2014-11-17 1291 }
fe4b991dcd84e0 Will Deacon 2014-11-17 1292 }
fe4b991dcd84e0 Will Deacon 2014-11-17 1293
fe4b991dcd84e0 Will Deacon 2014-11-17 1294 pr_info("selftest: completed with %d PASS %d FAIL\n", pass, fail);
fe4b991dcd84e0 Will Deacon 2014-11-17 1295 return fail ? -EFAULT : 0;fe4b991dcd84e0 Will Deacon 2014-11-17 @1296 }
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
kernel test robot
build botRe: [PATCH v9 05/25] dept: Tie to Lockdep and IRQ tracing
In reply to Byungchul ParkThank you for the patch! Perhaps something to improve:
Hi Byungchul,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on tip/locking/core]
[also build test WARNING on tip/sched/core drm-misc/drm-misc-next linus/master v6.2-rc6 next-20230201] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Byungchul-Park/llist-Move-llist_-head-node-definition-to-types-h/20230131-164632 patch link: https://lore.kernel.org/r/1675154394-25598-6-git-send-email-max.byungchul.park%40gmail.com patch subject: [PATCH v9 05/25] dept: Tie to Lockdep and IRQ tracing config: x86_64-allyesconfig (https://download.01.org/0day-ci/archive/20230202/[email protected]/config) compiler: gcc-11 (Debian 11.3.0-8) 11.3.0 reproduce (this is a W=1 build): # https://github.com/intel-lab-lkp/linux/commit/ee619b0e7faff81e8b794e2adcdf9102d5bc83e8 git remote add linux-review https://github.com/intel-lab-lkp/linux git fetch --no-tags linux-review Byungchul-Park/llist-Move-llist_-head-node-definition-to-types-h/20230131-164632 git checkout ee619b0e7faff81e8b794e2adcdf9102d5bc83e8 # save the config file mkdir build_dir && cp config build_dir/.config make W=1 O=build_dir ARCH=x86_64 olddefconfig make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable | Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
vmlinux.o: warning: objtool: vmx_vcpu_enter_exit+0x164: call to context_tracking_guest_exit() leaves .noinstr.text section vmlinux.o: warning: objtool: svm_vcpu_enter_exit+0x85: call to context_tracking_guest_exit() leaves .noinstr.text section vmlinux.o: warning: objtool: do_machine_check+0xb1e: call to pentium_machine_check.constprop.0() leaves .noinstr.text section
>> vmlinux.o: warning: objtool: lockdep_hardirqs_on+0x12: call to dept_hardirqs_on_ip() leaves .noinstr.text section >> vmlinux.o: warning: objtool: lockdep_hardirqs_off+0x9: call to dept_hardirqs_off_ip() leaves .noinstr.text section
vmlinux.o: warning: objtool: lock_is_held_type+0x78: call to dept_hardirqs_off() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_nmi_enter+0x14: call to dept_off() leaves .noinstr.text section vmlinux.o: warning: objtool: irqentry_nmi_exit+0x5a: call to dept_on() leaves .noinstr.text section vmlinux.o: warning: objtool: ct_idle_exit+0x40: call to dept_hardirqs_off() leaves .noinstr.text section vmlinux.o: warning: objtool: check_stackleak_irqoff+0x2b: call to end_of_stack.isra.0() leaves .noinstr.text section
objdump-func vmlinux.o lockdep_hardirqs_on:
0000 000000000000b980 <lockdep_hardirqs_on>:
0000 b980: f3 0f 1e fa endbr64 0004 b984: 55 push %rbp 0005 b985: 53 push %rbx 0006 b986: 48 89 fb mov %rdi,%rbx 0009 b989: 65 48 8b 2c 25 00 00 00 00 mov %gs:0x0,%rbp b98e: R_X86_64_32S pcpu_hot 0012 b992: e8 00 00 00 00 call b997 <lockdep_hardirqs_on+0x17> b993: R_X86_64_PLT32 dept_hardirqs_on_ip-0x4 0017 b997: 8b 3d 00 00 00 00 mov 0x0(%rip),%edi # b99d <lockdep_hardirqs_on+0x1d> b999: R_X86_64_PC32 debug_locks-0x4 001d b99d: 85 ff test %edi,%edi 001f b99f: 0f 84 cd 00 00 00 je ba72 <lockdep_hardirqs_on+0xf2> 0025 b9a5: 65 8b 05 00 00 00 00 mov %gs:0x0(%rip),%eax # b9ac <lockdep_hardirqs_on+0x2c> b9a8: R_X86_64_PC32 pcpu_hot+0x4 002c b9ac: a9 00 00 f0 00 test $0xf00000,%eax 0031 b9b1: 75 63 jne ba16 <lockdep_hardirqs_on+0x96> 0033 b9b3: 65 8b 05 00 00 00 00 mov %gs:0x0(%rip),%eax # b9ba <lockdep_hardirqs_on+0x3a> b9b6: R_X86_64_PC32 lockdep_recursion-0x4 003a b9ba: 85 c0 test %eax,%eax 003c b9bc: 0f 85 b0 00 00 00 jne ba72 <lockdep_hardirqs_on+0xf2> 0042 b9c2: 65 8b 05 00 00 00 00 mov %gs:0x0(%rip),%eax # b9c9 <lockdep_hardirqs_on+0x49> b9c5: R_X86_64_PC32 hardirqs_enabled-0x4 0049 b9c9: 85 c0 test %eax,%eax 004b b9cb: 0f 85 b2 00 00 00 jne ba83 <lockdep_hardirqs_on+0x103> 0051 b9d1: 8b 35 00 00 00 00 mov 0x0(%rip),%esi # b9d7 <lockdep_hardirqs_on+0x57> b9d3: R_X86_64_PC32 oops_in_progress-0x4 0057 b9d7: 85 f6 test %esi,%esi 0059 b9d9: 75 3b jne ba16 <lockdep_hardirqs_on+0x96> 005b b9db: 48 83 3d 00 00 00 00 00 cmpq $0x0,0x0(%rip) # b9e3 <lockdep_hardirqs_on+0x63> b9de: R_X86_64_PC32 pv_ops+0xeb 0063 b9e3: 0f 84 ed 00 00 00 je bad6 <lockdep_hardirqs_on+0x156> 0069 b9e9: ff 15 00 00 00 00 call *0x0(%rip) # b9ef <lockdep_hardirqs_on+0x6f> b9eb: R_X86_64_PC32 pv_ops+0xec 006f b9ef: f6 c4 02 test $0x2,%ah 0072 b9f2: 0f 85 c1 00 00 00 jne bab9 <lockdep_hardirqs_on+0x139> 0078 b9f8: 8b 15 00 00 00 00 mov 0x0(%rip),%edx # b9fe <lockdep_hardirqs_on+0x7e> b9fa: R_X86_64_PC32 oops_in_progress-0x4 007e b9fe: 85 d2 test %edx,%edx 0080 ba00: 75 14 jne ba16 <lockdep_hardirqs_on+0x96> 0082 ba02: 48 8b 85 c8 0f 00 00 mov 0xfc8(%rbp),%rax 0089 ba09: 48 39 85 f8 25 00 00 cmp %rax,0x25f8(%rbp) 0090 ba10: 0f 85 89 00 00 00 jne ba9f <lockdep_hardirqs_on+0x11f> 0096 ba16: 48 c7 c7 00 00 00 00 mov $0x0,%rdi ba19: R_X86_64_32S .rodata.str1.1+0x16d93 009d ba1d: e8 00 00 00 00 call ba22 <lockdep_hardirqs_on+0xa2> ba1e: R_X86_64_PLT32 __this_cpu_preempt_check-0x4 00a2 ba22: 8b 85 b0 10 00 00 mov 0x10b0(%rbp),%eax 00a8 ba28: 48 89 9d b8 10 00 00 mov %rbx,0x10b8(%rbp) 00af ba2f: 65 c7 05 00 00 00 00 01 00 00 00 movl $0x1,%gs:0x0(%rip) # ba3a <lockdep_hardirqs_on+0xba> ba32: R_X86_64_PC32 hardirqs_enabled-0x8 00ba ba3a: 83 c0 01 add $0x1,%eax 00bd ba3d: 48 83 3d 00 00 00 00 00 cmpq $0x0,0x0(%rip) # ba45 <lockdep_hardirqs_on+0xc5> ba40: R_X86_64_PC32 pv_ops+0xeb 00c5 ba45: 89 85 b0 10 00 00 mov %eax,0x10b0(%rbp) 00cb ba4b: 89 85 c8 10 00 00 mov %eax,0x10c8(%rbp) 00d1 ba51: 74 49 je ba9c <lockdep_hardirqs_on+0x11c> 00d3 ba53: ff 15 00 00 00 00 call *0x0(%rip) # ba59 <lockdep_hardirqs_on+0xd9> ba55: R_X86_64_PC32 pv_ops+0xec 00d9 ba59: f6 c4 02 test $0x2,%ah 00dc ba5c: 75 72 jne bad0 <lockdep_hardirqs_on+0x150> 00de ba5e: 48 c7 c7 00 00 00 00 mov $0x0,%rdi ba61: R_X86_64_32S .rodata.str1.1+0x16d99 00e5 ba65: e8 00 00 00 00 call ba6a <lockdep_hardirqs_on+0xea> ba66: R_X86_64_PLT32 __this_cpu_preempt_check-0x4 00ea ba6a: 65 48 ff 05 00 00 00 00 incq %gs:0x0(%rip) # ba72 <lockdep_hardirqs_on+0xf2> ba6e: R_X86_64_PC32 lockdep_stats+0xc 00f2 ba72: 5b pop %rbx 00f3 ba73: 5d pop %rbp 00f4 ba74: 31 c0 xor %eax,%eax 00f6 ba76: 31 d2 xor %edx,%edx 00f8 ba78: 31 c9 xor %ecx,%ecx 00fa ba7a: 31 f6 xor %esi,%esi 00fc ba7c: 31 ff xor %edi,%edi 00fe ba7e: e9 00 00 00 00 jmp ba83 <lockdep_hardirqs_on+0x103> ba7f: R_X86_64_PLT32 __x86_return_thunk-0x4 0103 ba83: 65 48 ff 05 00 00 00 00 incq %gs:0x0(%rip) # ba8b <lockdep_hardirqs_on+0x10b> ba87: R_X86_64_PC32 lockdep_stats+0x1c 010b ba8b: 5b pop %rbx 010c ba8c: 5d pop %rbp 010d ba8d: 31 c0 xor %eax,%eax 010f ba8f: 31 d2 xor %edx,%edx 0111 ba91: 31 c9 xor %ecx,%ecx 0113 ba93: 31 f6 xor %esi,%esi 0115 ba95: 31 ff xor %edi,%edi 0117 ba97: e9 00 00 00 00 jmp ba9c <lockdep_hardirqs_on+0x11c> ba98: R_X86_64_PLT32 __x86_return_thunk-0x4 011c ba9c: 90 nop 011d ba9d: 0f 0b ud2 011f ba9f: 90 nop 0120 baa0: e8 00 00 00 00 call baa5 <lockdep_hardirqs_on+0x125> baa1: R_X86_64_PLT32 debug_locks_off-0x4 0125 baa5: 85 c0 test %eax,%eax 0127 baa7: 74 0a je bab3 <lockdep_hardirqs_on+0x133> 0129 baa9: 8b 05 00 00 00 00 mov 0x0(%rip),%eax # baaf <lockdep_hardirqs_on+0x12f> baab: R_X86_64_PC32 debug_locks_silent-0x4 012f baaf: 85 c0 test %eax,%eax 0131 bab1: 74 41 je baf4 <lockdep_hardirqs_on+0x174> 0133 bab3: 90 nop 0134 bab4: e9 5d ff ff ff jmp ba16 <lockdep_hardirqs_on+0x96> 0139 bab9: 90 nop 013a baba: e8 00 00 00 00 call babf <lockdep_hardirqs_on+0x13f> babb: R_X86_64_PLT32 debug_locks_off-0x4 013f babf: 85 c0 test %eax,%eax 0141 bac1: 74 0a je bacd <lockdep_hardirqs_on+0x14d> 0143 bac3: 8b 0d 00 00 00 00 mov 0x0(%rip),%ecx # bac9 <lockdep_hardirqs_on+0x149> bac5: R_X86_64_PC32 debug_locks_silent-0x4 0149 bac9: 85 c9 test %ecx,%ecx 014b bacb: 74 0c je bad9 <lockdep_hardirqs_on+0x159> 014d bacd: 90 nop 014e bace: eb a2 jmp ba72 <lockdep_hardirqs_on+0xf2> 0150 bad0: 90 nop 0151 bad1: 0f 0b ud2 0153 bad3: 90 nop 0154 bad4: eb 88 jmp ba5e <lockdep_hardirqs_on+0xde> 0156 bad6: 90 nop 0157 bad7: 0f 0b ud2 0159 bad9: 90 nop 015a bada: 48 c7 c6 00 00 00 00 mov $0x0,%rsi badd: R_X86_64_32S .rodata.str1.1+0x16d82 0161 bae1: 48 c7 c7 00 00 00 00 mov $0x0,%rdi bae4: R_X86_64_32S .rodata.str1.1+0x16d41 0168 bae8: e8 00 00 00 00 call baed <lockdep_hardirqs_on+0x16d> bae9: R_X86_64_PLT32 __warn_printk-0x4 016d baed: 90 nop 016e baee: 0f 0b ud2 0170 baf0: 90 nop 0171 baf1: 90 nop 0172 baf2: eb d9 jmp bacd <lockdep_hardirqs_on+0x14d> 0174 baf4: 90 nop 0175 baf5: 48 c7 c6 00 00 00 00 mov $0x0,%rsi baf8: R_X86_64_32S .rodata.str1.8+0x2f280 017c bafc: 48 c7 c7 00 00 00 00 mov $0x0,%rdi baff: R_X86_64_32S .rodata.str1.1+0x16d41 0183 bb03: e8 00 00 00 00 call bb08 <lockdep_hardirqs_on+0x188> bb04: R_X86_64_PLT32 __warn_printk-0x4 0188 bb08: 90 nop 0189 bb09: 0f 0b ud2 018b bb0b: 90 nop 018c bb0c: 90 nop 018d bb0d: eb a4 jmp bab3 <lockdep_hardirqs_on+0x133> 018f bb0f: 66 66 2e 0f 1f 84 00 00 00 00 00 data16 cs nopw 0x0(%rax,%rax,1) 019a bb1a: 66 66 2e 0f 1f 84 00 00 00 00 00 data16 cs nopw 0x0(%rax,%rax,1) 01a5 bb25: 66 66 2e 0f 1f 84 00 00 00 00 00 data16 cs nopw 0x0(%rax,%rax,1) 01b0 bb30: 66 66 2e 0f 1f 84 00 00 00 00 00 data16 cs nopw 0x0(%rax,%rax,1) 01bb bb3b: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
PATCH v9 6/25 dept: Add proc knobs to show stats and dependency graph
In reply to Byungchul ParkIt'd be useful to show Dept internal stats and dependency graph on runtime via proc for better information. Introduced the knobs.
--- kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 24 ++++------ kernel/dependency/dept_internal.h | 26 +++++++++++ kernel/dependency/dept_proc.c | 95 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 15 deletions(-) create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_proc.c
kernel/dependency/Makefile +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DEPT) += dept.o +obj-$(CONFIG_DEPT) += dept_proc.o
kernel/dependency/dept.c +9 -15
@@ -74,6 +74,7 @@#include <linux/dept.h> #include <linux/utsname.h> #include <linux/kernel.h> +#include "dept_internal.h" static int dept_stop;static int dept_per_cpu_ready;
@@ -261,20 +262,13 @@ static inline bool valid_key(struct dept_key *k)* have been freed will be placed. */ -enum object_t { -#define OBJECT(id, nr) OBJECT_##id, -#include "dept_object.h" -#undef OBJECT - OBJECT_NR, -}; -#define OBJECT (id, nr) \ static struct dept_##id spool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id);#include "dept_object.h" #undef OBJECT -static struct dept_pool pool[OBJECT_NR] = { +struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \
@@ -304,7 +298,7 @@ static void *from_pool(enum object_t t)if (DEPT_WARN_ON (!irqs_disabled ()))return NULL ; - p = &pool[t]; + p = &dept_pool[t];/* * Try local pool first.
@@ -339,7 +333,7 @@ static void *from_pool(enum object_t t)static void to_pool (void *o,enum object_t t) { -struct dept_pool *p = &pool[t]; +struct dept_pool *p = &dept_pool[t];struct llist_head *h;preempt_disable ();
@@ -2136,7 +2130,7 @@ void dept_map_copy(struct dept_map *to, struct dept_map *from)clean_classes_cache (&to->map_key); } -static LIST_HEAD (classes); +LIST_HEAD (dept_classes);static inline bool within (const void *addr,void *start,unsigned long size) {
@@ -2168,7 +2162,7 @@ void dept_free_range(void *start, unsigned int sz)while (unlikely (!dept_lock ()))cpu_relax (); -list_for_each_entry_safe (c, n, &classes, all_node) { +list_for_each_entry_safe (c, n, &dept_classes, all_node) {if (!within ((void *)c->key, start, sz) && !within (c->name, start, sz))continue ;
@@ -2244,7 +2238,7 @@ static struct dept_class *check_new_class(struct dept_key *local, c->sub_id = sub_id; c->key = (unsigned long )(k->base + sub_id);hash_add_class (c); -list_add (&c->all_node, &classes); +list_add (&c->all_node, &dept_classes); unlock:dept_unlock (); caching:
@@ -2958,8 +2952,8 @@ static void migrate_per_cpu_pool(void)struct llist_head *from;struct llist_head *to; - from = &pool[i].boot_pool; - to =per_cpu_ptr (pool[i].lpool, boot_cpu); + from = &dept_pool[i].boot_pool; + to =per_cpu_ptr (dept_pool[i].lpool, boot_cpu);move_llist (to, from); } }
kernel/dependency/dept_internal.h new file +26 -0
@@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Dept(DEPendency Tracker) - runtime dependency tracker internal header +* +* Started by Byungchul Park <[email protected]>: +* +* Copyright (c) 2020 LG Electronics, Inc., Byungchul Park +*/ + +#ifndef __DEPT_INTERNAL_H +#define __DEPT_INTERNAL_H + +#ifdef CONFIG_DEPT + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, +#include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +extern struct list_head dept_classes; +extern struct dept_pool dept_pool[]; + +#endif +#endif /* __DEPT_INTERNAL_H */
kernel/dependency/dept_proc.c new file +95 -0
95 lines not on this page · raw message
Signature · 1 line
1.9.1
PATCH v9 7/25 dept: Apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete()
In reply to Byungchul ParkMakes Dept able to track dependencies by
wait_for_completion()/complete().
--- include/linux/completion.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-)
include/linux/completion.h +25 -5
@@ -10,6 +10,7 @@*/ #include <linux/swait.h> +#include <linux/dept_sdt.h> /* * struct completion - structure used to maintain state for a "completion"
@@ -26,14 +27,33 @@struct completion {unsigned int done;struct swait_queue_head wait; +struct dept_map dmap; }; +#define init_completion (x) \ +do { \ + sdt_map_init(&(x)->dmap); \ + __init_completion(x); \ +} while (0) + +/* +* XXX: No use cases for now. Fill the body when needed. +*/ #define init_completion_map (x, m) init_completion(x) -static inline void complete_acquire (struct completion *x) {} -static inline void complete_release (struct completion *x) {} + +static inline void complete_acquire (struct completion *x) +{ +sdt_might_sleep_start (&x->dmap); +} + +static inline void complete_release (struct completion *x) +{ +sdt_might_sleep_end (); +}#define COMPLETION_INITIALIZER (work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ + .dmap = DEPT_MAP_INITIALIZER(work, NULL), }#define COMPLETION_INITIALIZER_ONSTACK_MAP (work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); }))
@@ -75,13 +95,13 @@ static inline void complete_release(struct completion *x) {}#endif /** -* init_completion - Initialize a dynamically allocated completion +* __init_completion - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized * * This inline function will initialize a dynamically created completion * structure. */ -static inline void init_completion (struct completion *x) +static inline void __init_completion (struct completion *x) { x->done =0 ;init_swait_queue_head (&x->wait);
Signature · 1 line
1.9.1
kernel test robot
build botRe: [PATCH v9 07/25] dept: Apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete()
Thank you for the patch! Perhaps something to improve:
Hi Byungchul,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on tip/locking/core]
[also build test WARNING on tip/sched/core drm-misc/drm-misc-next linus/master v6.2-rc6 next-20230131] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Byungchul-Park/llist-Move-llist_-head-node-definition-to-types-h/20230131-164632 patch link: https://lore.kernel.org/r/1675154394-25598-8-git-send-email-max.byungchul.park%40gmail.com patch subject: [PATCH v9 07/25] dept: Apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() config: arc-allyesconfig (https://download.01.org/0day-ci/archive/20230201/[email protected]/config) compiler: arceb-elf-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/4773cd9b4e467e974a08cfcd690019b170f2e123
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Byungchul-Park/llist-Move-llist_-head-node-definition-to-types-h/20230131-164632 git checkout 4773cd9b4e467e974a08cfcd690019b170f2e123
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arc olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash drivers/scsi/qla2xxx/
If you fix the issue, kindly add following tag where applicable | Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
drivers/scsi/qla2xxx/qla_dfs.c: In function 'qla2x00_dfs_tgt_port_database_show':
>> drivers/scsi/qla2xxx/qla_dfs.c:227:1: warning: the frame size of 1032 bytes is larger than 1024 bytes [-Wframe-larger-than=]
227 | }
| ^vim +227 drivers/scsi/qla2xxx/qla_dfs.c
36c7845282eef0 Quinn Tran 2016-02-04 174
c423437e3ff41b Himanshu Madhani 2017-03-15 175 static int
c423437e3ff41b Himanshu Madhani 2017-03-15 176 qla2x00_dfs_tgt_port_database_show(struct seq_file *s, void *unused)
c423437e3ff41b Himanshu Madhani 2017-03-15 177 {
c423437e3ff41b Himanshu Madhani 2017-03-15 178 scsi_qla_host_t *vha = s->private;
c423437e3ff41b Himanshu Madhani 2017-03-15 179 struct qla_hw_data *ha = vha->hw;
4e5a05d1ecd92c Arun Easi 2020-09-03 180 struct gid_list_info *gid_list;
c423437e3ff41b Himanshu Madhani 2017-03-15 181 dma_addr_t gid_list_dma;
c423437e3ff41b Himanshu Madhani 2017-03-15 182 fc_port_t fc_port;
4e5a05d1ecd92c Arun Easi 2020-09-03 183 char *id_iter;
c423437e3ff41b Himanshu Madhani 2017-03-15 184 int rc, i;
c423437e3ff41b Himanshu Madhani 2017-03-15 185 uint16_t entries, loop_id;
c423437e3ff41b Himanshu Madhani 2017-03-15 186
c423437e3ff41b Himanshu Madhani 2017-03-15 187 seq_printf(s, "%s\n", vha->host_str);
c423437e3ff41b Himanshu Madhani 2017-03-15 188 gid_list = dma_alloc_coherent(&ha->pdev->dev,
c423437e3ff41b Himanshu Madhani 2017-03-15 189 qla2x00_gid_list_size(ha),
c423437e3ff41b Himanshu Madhani 2017-03-15 190 &gid_list_dma, GFP_KERNEL);
c423437e3ff41b Himanshu Madhani 2017-03-15 191 if (!gid_list) {
83548fe2fcbb78 Quinn Tran 2017-06-02 192 ql_dbg(ql_dbg_user, vha, 0x7018,
c423437e3ff41b Himanshu Madhani 2017-03-15 193 "DMA allocation failed for %u\n",
c423437e3ff41b Himanshu Madhani 2017-03-15 194 qla2x00_gid_list_size(ha));
c423437e3ff41b Himanshu Madhani 2017-03-15 195 return 0;
c423437e3ff41b Himanshu Madhani 2017-03-15 196 }
c423437e3ff41b Himanshu Madhani 2017-03-15 197
c423437e3ff41b Himanshu Madhani 2017-03-15 198 rc = qla24xx_gidlist_wait(vha, gid_list, gid_list_dma,
c423437e3ff41b Himanshu Madhani 2017-03-15 199 &entries);
c423437e3ff41b Himanshu Madhani 2017-03-15 200 if (rc != QLA_SUCCESS)
c423437e3ff41b Himanshu Madhani 2017-03-15 201 goto out_free_id_list;
c423437e3ff41b Himanshu Madhani 2017-03-15 202
4e5a05d1ecd92c Arun Easi 2020-09-03 203 id_iter = (char *)gid_list;
c423437e3ff41b Himanshu Madhani 2017-03-15 204
c423437e3ff41b Himanshu Madhani 2017-03-15 205 seq_puts(s, "Port Name Port ID Loop ID\n");
c423437e3ff41b Himanshu Madhani 2017-03-15 206
c423437e3ff41b Himanshu Madhani 2017-03-15 207 for (i = 0; i < entries; i++) {
4e5a05d1ecd92c Arun Easi 2020-09-03 208 struct gid_list_info *gid =
4e5a05d1ecd92c Arun Easi 2020-09-03 209 (struct gid_list_info *)id_iter;
c423437e3ff41b Himanshu Madhani 2017-03-15 210 loop_id = le16_to_cpu(gid->loop_id);
c423437e3ff41b Himanshu Madhani 2017-03-15 211 memset(&fc_port, 0, sizeof(fc_port_t));
c423437e3ff41b Himanshu Madhani 2017-03-15 212
c423437e3ff41b Himanshu Madhani 2017-03-15 213 fc_port.loop_id = loop_id;
c423437e3ff41b Himanshu Madhani 2017-03-15 214
c423437e3ff41b Himanshu Madhani 2017-03-15 215 rc = qla24xx_gpdb_wait(vha, &fc_port, 0);
c423437e3ff41b Himanshu Madhani 2017-03-15 216 seq_printf(s, "%8phC %02x%02x%02x %d\n",
c423437e3ff41b Himanshu Madhani 2017-03-15 217 fc_port.port_name, fc_port.d_id.b.domain,
c423437e3ff41b Himanshu Madhani 2017-03-15 218 fc_port.d_id.b.area, fc_port.d_id.b.al_pa,
c423437e3ff41b Himanshu Madhani 2017-03-15 219 fc_port.loop_id);
4e5a05d1ecd92c Arun Easi 2020-09-03 220 id_iter += ha->gid_list_info_size;
c423437e3ff41b Himanshu Madhani 2017-03-15 221 }
c423437e3ff41b Himanshu Madhani 2017-03-15 222 out_free_id_list:
c423437e3ff41b Himanshu Madhani 2017-03-15 223 dma_free_coherent(&ha->pdev->dev, qla2x00_gid_list_size(ha),
c423437e3ff41b Himanshu Madhani 2017-03-15 224 gid_list, gid_list_dma);
c423437e3ff41b Himanshu Madhani 2017-03-15 225
c423437e3ff41b Himanshu Madhani 2017-03-15 226 return 0;c423437e3ff41b Himanshu Madhani 2017-03-15 @227 } c423437e3ff41b Himanshu Madhani 2017-03-15 228
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
PATCH v9 8/25 dept: Apply sdt_might_sleep_{start,end}() to PG_{locked,writeback} wait
In reply to Byungchul ParkMakes Dept able to track dependencies by PG_{locked,writeback} waits.
--- mm/filemap.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
mm/filemap.c +11 -0
@@ -42,6 +42,7 @@#include <linux/ramfs.h> #include <linux/page_idle.h> #include <linux/migrate.h> +#include <linux/dept_sdt.h> #include <asm/pgalloc.h> #include <asm/tlbflush.h> #include "internal.h"
@@ -1215,6 +1216,9 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr,/* How many times do we accept lock stealing from under a waiter? */ int sysctl_page_lock_unfairness =5 ; +static struct dept_map __maybe_unused PG_locked_map =DEPT_MAP_INITIALIZER (PG_locked_map,NULL ); +static struct dept_map __maybe_unused PG_writeback_map =DEPT_MAP_INITIALIZER (PG_writeback_map,NULL ); +static inline int folio_wait_bit_common (struct folio *folio,int bit_nr,int state,enum behavior behavior) {
@@ -1226,6 +1230,11 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,unsigned long pflags;bool in_thrashing; +if (bit_nr == PG_locked) +sdt_might_sleep_start (&PG_locked_map); +else if (bit_nr == PG_writeback) +sdt_might_sleep_start (&PG_writeback_map); +if (bit_nr == PG_locked && !folio_test_uptodate (folio) &&folio_test_workingset (folio)) {delayacct_thrashing_start (&in_thrashing);
@@ -1327,6 +1336,8 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,*/ finish_wait (q, wait); +sdt_might_sleep_end (); +if (thrashing) {delayacct_thrashing_end (&in_thrashing);psi_memstall_leave (&pflags);
Signature · 1 line
1.9.1
PATCH v9 9/25 dept: Apply sdt_might_sleep_{start,end}() to swait
In reply to Byungchul ParkMakes Dept able to track dependencies by swaits.
--- include/linux/swait.h | 3 +++ 1 file changed, 3 insertions(+)
include/linux/swait.h +3 -0
@@ -6,6 +6,7 @@#include <linux/stddef.h> #include <linux/spinlock.h> #include <linux/wait.h> +#include <linux/dept_sdt.h> #include <asm/current.h> /*
@@ -161,6 +162,7 @@ static inline bool swq_has_sleeper(struct swait_queue_head *wq)struct swait_queue __wait; \long __ret = ret; \ \ +sdt_might_sleep_start (NULL ); \INIT_LIST_HEAD (&__wait.task_list); \for (;;) { \long __int =prepare_to_swait_event (&wq, &__wait, state);\
@@ -176,6 +178,7 @@ static inline bool swq_has_sleeper(struct swait_queue_head *wq) cmd; \ } \finish_swait (&wq, &__wait); \ +sdt_might_sleep_end (); \ __out: __ret; \ })
Signature · 1 line
1.9.1
PATCH v9 10/25 dept: Apply sdt_might_sleep_{start,end}() to waitqueue wait
In reply to Byungchul ParkMakes Dept able to track dependencies by waitqueue waits.
--- include/linux/wait.h | 3 +++ 1 file changed, 3 insertions(+)
include/linux/wait.h +3 -0
@@ -7,6 +7,7 @@#include <linux/list.h> #include <linux/stddef.h> #include <linux/spinlock.h> +#include <linux/dept_sdt.h> #include <asm/current.h> #include <uapi/linux/wait.h>
@@ -303,6 +304,7 @@ static inline void wake_up_pollfree(struct wait_queue_head *wq_head)struct wait_queue_entry __wq_entry; \long __ret = ret;/* explicit shadow */ \ \ +sdt_might_sleep_start (NULL ); \init_wait_entry (&__wq_entry, exclusive ?WQ_FLAG_EXCLUSIVE :0 ); \for (;;) { \long __int =prepare_to_wait_event (&wq_head, &__wq_entry, state);\
@@ -318,6 +320,7 @@ static inline void wake_up_pollfree(struct wait_queue_head *wq_head) cmd; \ } \finish_wait (&wq_head, &__wq_entry); \ +sdt_might_sleep_end (); \ __out: __ret; \ })
Signature · 1 line
1.9.1
PATCH v9 11/25 dept: Apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait
In reply to Byungchul ParkMakes Dept able to track dependencies by hashed-waitqueue waits.
--- include/linux/wait_bit.h | 3 +++ 1 file changed, 3 insertions(+)
include/linux/wait_bit.h +3 -0
@@ -6,6 +6,7 @@* Linux wait-bit related types and methods: */ #include <linux/wait.h> +#include <linux/dept_sdt.h> struct wait_bit_key {void *flags;
@@ -246,6 +247,7 @@ struct wait_bit_queue_entry {struct wait_bit_queue_entry __wbq_entry; \long __ret = ret;/* explicit shadow */ \ \ +sdt_might_sleep_start (NULL ); \init_wait_var_entry (&__wbq_entry, var, \ exclusive ?WQ_FLAG_EXCLUSIVE :0 ); \for (;;) { \
@@ -263,6 +265,7 @@ struct wait_bit_queue_entry { cmd; \ } \finish_wait (__wq_head, &__wbq_entry.wq_entry); \ +sdt_might_sleep_end (); \ __out: __ret; \ })
Signature · 1 line
1.9.1
PATCH v9 12/25 dept: Distinguish each syscall context from another
In reply to Byungchul ParkIt enters kernel mode on each syscall and each syscall handling should be considered independently from the point of view of Dept. Otherwise, Dept may wrongly track dependencies across different syscalls.
That might be a real dependency from user mode. However, now that Dept just started to work, conservatively let Dept not track dependencies across different syscalls.
--- arch/arm64/kernel/syscall.c | 2 ++ arch/x86/entry/common.c | 4 +++ include/linux/dept.h | 39 +++++++++++++++----------- kernel/dependency/dept.c | 67 +++++++++++++++++++++++---------------------- 4 files changed, 63 insertions(+), 49 deletions(-)
arch/arm64/kernel/syscall.c +2 -0
@@ -7,6 +7,7 @@#include <linux/ptrace.h> #include <linux/randomize_kstack.h> #include <linux/syscalls.h> +#include <linux/dept.h> #include <asm/daifflags.h> #include <asm/debug-monitors.h>
@@ -105,6 +106,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr,*/ local_daif_restore (DAIF_PROCCTX ); +dept_kernel_enter ();if (flags & _TIF_MTE_ASYNC_FAULT) {/*
arch/x86/entry/common.c +4 -0
@@ -19,6 +19,7 @@#include <linux/nospec.h> #include <linux/syscalls.h> #include <linux/uaccess.h> +#include <linux/dept.h> #ifdef CONFIG_XEN_PV #include <xen/xen-ops.h>
@@ -72,6 +73,7 @@ static __always_inline bool do_syscall_x32(struct pt_regs *regs, int nr) __visible noinstr void do_syscall_64(struct pt_regs *regs, int nr) { +dept_kernel_enter ();add_random_kstack_offset (); nr =syscall_enter_from_user_mode (regs, nr);
@@ -120,6 +122,7 @@ __visible noinstr void do_int80_syscall_32(struct pt_regs *regs) {int nr =syscall_32_enter (regs); +dept_kernel_enter ();add_random_kstack_offset ();/* * Subtlety here: if ptrace pokes something larger than 2^31-1 into
@@ -140,6 +143,7 @@ static noinstr bool __do_fast_syscall_32(struct pt_regs *regs)int nr =syscall_32_enter (regs);int res; +dept_kernel_enter ();add_random_kstack_offset ();/* * This cannot use syscall_enter_from_user_mode() as it has to
include/linux/dept.h +23 -16
@@ -25,11 +25,16 @@#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT)#define DEPT_MAX_SUBCLASSES_CACHE 2 -#define DEPT_SIRQ 0 -#define DEPT_HIRQ 1 -#define DEPT_IRQS_NR 2 -#define DEPT_SIRQF (1UL << DEPT_SIRQ) -#define DEPT_HIRQF (1UL << DEPT_HIRQ) +enum { +DEPT_CXT_SIRQ =0 , +DEPT_CXT_HIRQ , +DEPT_CXT_IRQS_NR , +DEPT_CXT_PROCESS =DEPT_CXT_IRQS_NR , +DEPT_CXTS_NR +}; + +#define DEPT_SIRQF (1UL << DEPT_CXT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_CXT_HIRQ)struct dept_ecxt ;struct dept_iecxt {
@@ -94,8 +99,8 @@ struct dept_class {/* * for tracking IRQ dependencies */ -struct dept_iecxt iecxt[DEPT_IRQS_NR ]; -struct dept_iwait iwait[DEPT_IRQS_NR ]; +struct dept_iecxt iecxt[DEPT_CXT_IRQS_NR ]; +struct dept_iwait iwait[DEPT_CXT_IRQS_NR ];/* * classified by a map embedded in task_struct,
@@ -207,8 +212,8 @@ struct dept_ecxt {/* * where the IRQ-enabled happened */ -unsigned long enirq_ip[DEPT_IRQS_NR ]; -struct dept_stack *enirq_stack[DEPT_IRQS_NR ]; +unsigned long enirq_ip[DEPT_CXT_IRQS_NR ]; +struct dept_stack *enirq_stack[DEPT_CXT_IRQS_NR ];/* * where the event context started
@@ -252,8 +257,8 @@ struct dept_wait {/* * where the IRQ wait happened */ -unsigned long irq_ip[DEPT_IRQS_NR ]; -struct dept_stack *irq_stack[DEPT_IRQS_NR ]; +unsigned long irq_ip[DEPT_CXT_IRQS_NR ]; +struct dept_stack *irq_stack[DEPT_CXT_IRQS_NR ];/* * where the wait happened
@@ -406,19 +411,19 @@ struct dept_task {int wait_hist_pos;/* -* sequential id to identify each IRQ context +* sequential id to identify each context */ -unsigned int irq_id[DEPT_IRQS_NR ]; +unsigned int cxt_id[DEPT_CXTS_NR ];/* * for tracking IRQ-enabled points with cross-event */ -unsigned int wgen_enirq[DEPT_IRQS_NR ]; +unsigned int wgen_enirq[DEPT_CXT_IRQS_NR ];/* * for keeping up-to-date IRQ-enabled points */ -unsigned long enirq_ip[DEPT_IRQS_NR ]; +unsigned long enirq_ip[DEPT_CXT_IRQS_NR ];/* * current effective IRQ-enabled flag
@@ -470,7 +475,7 @@ struct dept_task { .wait_hist = { { .wait = NULL, } }, \ .ecxt_held_pos = 0, \ .wait_hist_pos = 0, \ - .irq_id = { 0U }, \ + .cxt_id = { 0U }, \ .wgen_enirq = { 0U }, \ .enirq_ip = { 0UL }, \ .eff_enirqf = 0UL, \
@@ -509,6 +514,7 @@ struct dept_task {extern void dept_ecxt_exit (struct dept_map *m,unsigned long e_f,unsigned long ip);extern void dept_sched_enter (void );extern void dept_sched_exit (void ); +extern void dept_kernel_enter (void );static inline void dept_ecxt_enter_nokeep (struct dept_map *m) {
@@ -560,6 +566,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m)#define dept_ecxt_exit (m, e_f, ip) do { } while (0)#define dept_sched_enter () do { } while (0)#define dept_sched_exit () do { } while (0) +#define dept_kernel_enter () do { } while (0)#define dept_ecxt_enter_nokeep (m) do { } while (0)#define dept_key_init (k) do { (void)(k); } while (0)#define dept_key_destroy (k) do { (void)(k); } while (0)
kernel/dependency/dept.c +34 -33
197 lines not on this page · raw message
Signature · 1 line
1.9.1
PATCH v9 13/25 dept: Distinguish each work from another
In reply to Byungchul ParkWorkqueue already provides concurrency control. By that, any wait in a work doesn't prevents events in other works with the control enabled. Thus, each work would better be considered a different context.
So let Dept assign a different context id to each work.
--- include/linux/dept.h | 2 ++ kernel/dependency/dept.c | 10 ++++++++++ kernel/workqueue.c | 3 +++ 3 files changed, 15 insertions(+)
include/linux/dept.h +2 -0
@@ -515,6 +515,7 @@ struct dept_task {extern void dept_sched_enter (void );extern void dept_sched_exit (void );extern void dept_kernel_enter (void ); +extern void dept_work_enter (void );static inline void dept_ecxt_enter_nokeep (struct dept_map *m) {
@@ -567,6 +568,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m)#define dept_sched_enter () do { } while (0)#define dept_sched_exit () do { } while (0)#define dept_kernel_enter () do { } while (0) +#define dept_work_enter () do { } while (0)#define dept_ecxt_enter_nokeep (m) do { } while (0)#define dept_key_init (k) do { (void)(k); } while (0)#define dept_key_destroy (k) do { (void)(k); } while (0)
kernel/dependency/dept.c +10 -0
@@ -1977,6 +1977,16 @@ void dept_hardirqs_off_ip(unsigned long ip) }EXPORT_SYMBOL_GPL (dept_hardirqs_off_ip); +/* +* Assign a different context id to each work. +*/ +void dept_work_enter (void ) +{ +struct dept_task *dt =dept_task (); + + dt->cxt_id[DEPT_CXT_PROCESS ] +=1UL <<DEPT_CXTS_NR ; +} +void dept_kernel_enter (void ) {struct dept_task *dt =dept_task ();
kernel/workqueue.c +3 -0
@@ -51,6 +51,7 @@#include <linux/sched/isolation.h> #include <linux/nmi.h> #include <linux/kvm_para.h> +#include <linux/dept.h> #include "workqueue_internal.h"
@@ -2199,6 +2200,8 @@ static void process_one_work(struct worker *worker, struct work_struct *work)lockdep_copy_map (&lockdep_map, &work->lockdep_map);#endif +dept_work_enter (); +/* ensure we're on the correct CPU */ WARN_ON_ONCE (!(pool->flags &POOL_DISASSOCIATED ) &&raw_smp_processor_id () != pool->cpu);
Signature · 1 line
1.9.1
PATCH v9 14/25 dept: Add a mechanism to refill the internal memory pools on running out
In reply to Byungchul ParkDept engine works in a constrained environment. For example, Dept cannot make use of dynamic allocation e.g. kmalloc(). So Dept has been using static pools to keep memory chunks Dept uses.
However, Dept would barely work once any of the pools gets run out. So implemented a mechanism for the refill on the lack by any chance, using irq work and workqueue that fits on the contrained environment.
--- include/linux/dept.h | 19 ++++++-- kernel/dependency/dept.c | 104 ++++++++++++++++++++++++++++++++++------ kernel/dependency/dept_object.h | 10 ++-- kernel/dependency/dept_proc.c | 8 ++-- 4 files changed, 112 insertions(+), 29 deletions(-)
include/linux/dept.h +15 -4
@@ -336,9 +336,19 @@ struct dept_pool {size_t obj_sz;/* -* the number of the static array +* the remaining number of the object in spool */ -atomic_t obj_nr; +int obj_nr; + +/* +* the number of the object in spool +*/ +int tot_nr; + +/* +* accumulated amount of memory used by the object in byte +*/ +atomic_t acc_sz;/* * offset of ->pool_node
@@ -348,9 +358,10 @@ struct dept_pool {/* * pointer to the pool */ -void *spool; +void *spool;/* static pool */ +void *rpool;/* reserved pool */ struct llist_head boot_pool; -struct llist_head __percpu *lpool; +struct llist_head __percpu *lpool;/* local pool */ };struct dept_ecxt_held {
kernel/dependency/dept.c +89 -15
158 lines not on this page · raw message
kernel/dependency/dept_object.h +5 -5
@@ -6,8 +6,8 @@* nr: # of the object that should be kept in the pool. */ -OBJECT(dep, 1024 * 8) -OBJECT(class, 1024 * 8) -OBJECT(stack, 1024 * 32) -OBJECT(ecxt, 1024 * 16) -OBJECT (wait,1024 *32 ) +OBJECT(dep, 1024 * 4 * 2) +OBJECT(class, 1024 * 4) +OBJECT(stack, 1024 * 4 * 8) +OBJECT(ecxt, 1024 * 4 * 2) +OBJECT (wait,1024 *4 *4 )
kernel/dependency/dept_proc.c +3 -5
@@ -73,12 +73,10 @@ static int dept_stats_show(struct seq_file *m, void *v) {int r; -seq_puts (m,"Availability in the static pools:\n\n" ); +seq_puts (m,"Accumulated amount of memory used by pools:\n\n" );#define OBJECT (id, nr) \ - r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ - if (r < 0) \ - r = 0; \ - seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + r = atomic_read(&dept_pool[OBJECT_##id].acc_sz); \ + seq_printf(m, "%s\t%d KB\n", #id, r / 1024);#include "dept_object.h" #undef OBJECT
Signature · 1 line
1.9.1
PATCH v9 15/25 locking/lockdep, cpu/hotplus: Use a weaker annotation in AP thread
In reply to Byungchul Parkcb92173d1f0 ("locking/lockdep, cpu/hotplug: Annotate AP thread") was introduced to make lockdep_assert_cpus_held() work in AP thread.
However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that.
Furthermore, now that Dept was introduced, false positive alarms was reported by that. Replaced it with try lock annotation.
--- kernel/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/cpu.c +1 -1
@@ -356,7 +356,7 @@ int lockdep_is_cpus_held(void)static void lockdep_acquire_cpus_lock (void ) { -rwsem_acquire (&cpu_hotplug_lock.dep_map,0 ,0 , _THIS_IP_); +rwsem_acquire (&cpu_hotplug_lock.dep_map,0 ,1 , _THIS_IP_); }static void lockdep_release_cpus_lock (void )
Signature · 1 line
1.9.1
PATCH v9 16/25 dept: Apply sdt_might_sleep_{start,end}() to dma fence wait
In reply to Byungchul ParkMakes Dept able to track dma fence waits.
--- drivers/dma-buf/dma-fence.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/dma-buf/dma-fence.c +5 -0
@@ -16,6 +16,7 @@#include <linux/dma-fence.h> #include <linux/sched/signal.h> #include <linux/seq_file.h> +#include <linux/dept_sdt.h> #define CREATE_TRACE_POINTS #include <trace/events/dma_fence.h>
@@ -782,6 +783,7 @@ struct default_wait_cb { cb.task = current;list_add (&cb.base.node, &fence->cb_list); +sdt_might_sleep_start (NULL );while (!test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags) && ret >0 ) {if (intr)__set_current_state (TASK_INTERRUPTIBLE );
@@ -795,6 +797,7 @@ struct default_wait_cb {if (ret >0 && intr &&signal_pending (current)) ret = -ERESTARTSYS ; } +sdt_might_sleep_end ();if (!list_empty (&cb.base.node))list_del (&cb.base.node);
@@ -884,6 +887,7 @@ struct default_wait_cb { } } +sdt_might_sleep_start (NULL );while (ret >0 ) {if (intr)set_current_state (TASK_INTERRUPTIBLE );
@@ -898,6 +902,7 @@ struct default_wait_cb {if (ret >0 && intr &&signal_pending (current)) ret = -ERESTARTSYS ; } +sdt_might_sleep_end ();__set_current_state (TASK_RUNNING );
Signature · 1 line
1.9.1
PATCH v9 17/25 dept: Track timeout waits separately with a new Kconfig
In reply to Byungchul ParkWaits with valid timeouts don't actually cause deadlocks. However, Dept has been reporting the cases as well because it's worth informing the circular dependency for some cases where, for example, timeout is used to avoid a deadlock but not meant to be expired.
However, yes, there are also a lot of, even more, cases where timeout is used for its clear purpose and meant to be expired.
Let Dept report these as an information rather than shouting DEADLOCK. Plus, introduced CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT Kconfig to make it optional so that any reports involving waits with timeouts can be turned on/off depending on the purpose.
--- include/linux/dept.h | 15 ++++++++--- include/linux/dept_ldt.h | 6 ++--- include/linux/dept_sdt.h | 12 ++++++--- kernel/dependency/dept.c | 66 +++++++++++++++++++++++++++++++++++++++++------- lib/Kconfig.debug | 10 ++++++++ 5 files changed, 89 insertions(+), 20 deletions(-)
include/linux/dept.h +11 -4
@@ -270,6 +270,11 @@ struct dept_wait {* whether this wait is for commit in scheduler */ bool sched_sleep; + +/* +* whether a timeout is set +*/ +bool timeout; }; }; };
@@ -458,6 +463,7 @@ struct dept_task {bool stage_sched_map;const char *stage_w_fn;unsigned long stage_ip; +bool stage_timeout;/* * the number of missing ecxts
@@ -496,6 +502,7 @@ struct dept_task { .stage_sched_map = false, \ .stage_w_fn = NULL, \ .stage_ip = 0UL, \ + .stage_timeout = false, \ .missing_ecxt = 0, \ .hardirqs_enabled = false, \ .softirqs_enabled = false, \
@@ -513,8 +520,8 @@ struct dept_task {extern void dept_map_reinit (struct dept_map *m,struct dept_key *k,int sub_u,const char *n);extern void dept_map_copy (struct dept_map *to,struct dept_map *from); -extern void dept_wait (struct dept_map *m,unsigned long w_f,unsigned long ip,const char *w_fn,int sub_l); -extern void dept_stage_wait (struct dept_map *m,struct dept_key *k,unsigned long ip,const char *w_fn); +extern void dept_wait (struct dept_map *m,unsigned long w_f,unsigned long ip,const char *w_fn,int sub_l,long timeout); +extern void dept_stage_wait (struct dept_map *m,struct dept_key *k,unsigned long ip,const char *w_fn,long timeout);extern void dept_request_event_wait_commit (void );extern void dept_clean_stage (void );extern void dept_stage_event (struct task_struct *t,unsigned long ip);
@@ -566,8 +573,8 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m)#define dept_map_reinit (m, k, su, n) do { (void)(n); (void)(k); } while (0)#define dept_map_copy (t, f) do { } while (0) -#define dept_wait (m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) -#define dept_stage_wait (m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_wait (m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) +#define dept_stage_wait (m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); } while (0)#define dept_request_event_wait_commit () do { } while (0)#define dept_clean_stage () do { } while (0)#define dept_stage_event (t, ip) do { } while (0)
include/linux/dept_ldt.h +3 -3
@@ -27,7 +27,7 @@else if (t) \dept_ecxt_enter (m,LDT_EVT_L , i,"trylock" ,"unlock" , sl);\else { \ -dept_wait (m,LDT_EVT_L , i,"lock" , sl); \ +dept_wait (m,LDT_EVT_L , i,"lock" , sl, false); \dept_ecxt_enter (m,LDT_EVT_L , i,"lock" ,"unlock" , sl);\ } \ }while (0 )
@@ -39,7 +39,7 @@else if (t) \dept_ecxt_enter (m,LDT_EVT_R , i,"read_trylock" ,"read_unlock" , sl);\else { \ -dept_wait (m, q ?LDT_EVT_RW :LDT_EVT_W , i,"read_lock" , sl);\ +dept_wait (m, q ?LDT_EVT_RW :LDT_EVT_W , i,"read_lock" , sl, false);\dept_ecxt_enter (m,LDT_EVT_R , i,"read_lock" ,"read_unlock" , sl);\ } \ }while (0 )
@@ -51,7 +51,7 @@else if (t) \dept_ecxt_enter (m,LDT_EVT_W , i,"write_trylock" ,"write_unlock" , sl);\else { \ -dept_wait (m,LDT_EVT_RW , i,"write_lock" , sl); \ +dept_wait (m,LDT_EVT_RW , i,"write_lock" , sl, false);\dept_ecxt_enter (m,LDT_EVT_W , i,"write_lock" ,"write_unlock" , sl);\ } \ }while (0 )
include/linux/dept_sdt.h +8 -4
@@ -22,11 +22,12 @@#define sdt_map_init_key (m, k) dept_map_init(m, k, 0, #m) -#define sdt_wait (m) \ +#define sdt_wait_timeout (m, t) \ do { \ dept_request_event(m); \ - dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) +#define sdt_wait (m) sdt_wait_timeout(m, -1L)/* * sdt_might_sleep() and its family will be committed in __schedule()
@@ -37,12 +38,13 @@/* * Use the code location as the class key if an explicit map is not used. */ -#define sdt_might_sleep_start (m) \ +#define sdt_might_sleep_start_timeout (m, t) \ do { \ struct dept_map *__m = m; \ static struct dept_key __key; \ - dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__, t);\ } while (0) +#define sdt_might_sleep_start (m) sdt_might_sleep_start_timeout(m, -1L)#define sdt_might_sleep_end () dept_clean_stage()
@@ -52,7 +54,9 @@#else /* !CONFIG_DEPT */ #define sdt_map_init (m) do { } while (0)#define sdt_map_init_key (m, k) do { (void)(k); } while (0) +#define sdt_wait_timeout (m, t) do { } while (0)#define sdt_wait (m) do { } while (0) +#define sdt_might_sleep_start_timeout (m, t) do { } while (0)#define sdt_might_sleep_start (m) do { } while (0)#define sdt_might_sleep_end () do { } while (0)#define sdt_ecxt_enter (m) do { } while (0)
kernel/dependency/dept.c +57 -9
172 lines not on this page · raw message
lib/Kconfig.debug +10 -0
@@ -1282,6 +1282,16 @@ config DEPT noting, to mitigate the impact by the false positives, multi reporting has been supported. +config DEPT_AGGRESSIVE_TIMEOUT_WAIT + bool "Aggressively track even timeout waits" + depends on DEPT + default n + help + Timeout wait doesn't contribute to a deadlock. However, + informing a circular dependency might be helpful for cases + that timeout is used to avoid a deadlock. Say N if you'd like + to avoid verbose reports. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
Signature · 1 line
1.9.1
PATCH v9 18/25 dept: Apply timeout consideration to wait_for_completion()/complete()
In reply to Byungchul ParkNow that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to wait_for_completion()/complete().
--- include/linux/completion.h | 4 ++-- kernel/sched/completion.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
include/linux/completion.h +2 -2
@@ -41,9 +41,9 @@ struct completion {*/ #define init_completion_map (x, m) init_completion(x) -static inline void complete_acquire (struct completion *x) +static inline void complete_acquire (struct completion *x,long timeout) { -sdt_might_sleep_start (&x->dmap); +sdt_might_sleep_start_timeout (&x->dmap, timeout); }static inline void complete_release (struct completion *x)
kernel/sched/completion.c +1 -1
@@ -100,7 +100,7 @@ void complete_all(struct completion *x) {might_sleep (); -complete_acquire (x); +complete_acquire (x, timeout);raw_spin_lock_irq (&x->wait.lock); timeout =do_wait_for_common (x, action, timeout, state);
Signature · 1 line
1.9.1
PATCH v9 19/25 dept: Apply timeout consideration to swait
In reply to Byungchul ParkNow that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to swait, assuming an input 'ret' in ___swait_event() macro is used as a timeout value.
--- include/linux/swait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/swait.h +1 -1
@@ -162,7 +162,7 @@ static inline bool swq_has_sleeper(struct swait_queue_head *wq)struct swait_queue __wait; \long __ret = ret; \ \ -sdt_might_sleep_start (NULL ); \ +sdt_might_sleep_start_timeout (NULL , __ret); \INIT_LIST_HEAD (&__wait.task_list); \for (;;) { \long __int =prepare_to_swait_event (&wq, &__wait, state);\
Signature · 1 line
1.9.1
PATCH v9 20/25 dept: Apply timeout consideration to waitqueue wait
In reply to Byungchul ParkNow that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to waitqueue wait, assuming an input 'ret' in ___wait_event() macro is used as a timeout value.
--- include/linux/wait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/wait.h +1 -1
@@ -304,7 +304,7 @@ static inline void wake_up_pollfree(struct wait_queue_head *wq_head)struct wait_queue_entry __wq_entry; \long __ret = ret;/* explicit shadow */ \ \ -sdt_might_sleep_start (NULL ); \ +sdt_might_sleep_start_timeout (NULL , __ret); \init_wait_entry (&__wq_entry, exclusive ?WQ_FLAG_EXCLUSIVE :0 ); \for (;;) { \long __int =prepare_to_wait_event (&wq_head, &__wq_entry, state);\
Signature · 1 line
1.9.1
PATCH v9 21/25 dept: Apply timeout consideration to hashed-waitqueue wait
In reply to Byungchul ParkNow that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to hashed-waitqueue wait, assuming an input 'ret' in ___wait_var_event() macro is used as a timeout value.
--- include/linux/wait_bit.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/wait_bit.h +1 -1
@@ -247,7 +247,7 @@ struct wait_bit_queue_entry {struct wait_bit_queue_entry __wbq_entry; \long __ret = ret;/* explicit shadow */ \ \ -sdt_might_sleep_start (NULL ); \ +sdt_might_sleep_start_timeout (NULL , __ret); \init_wait_var_entry (&__wbq_entry, var, \ exclusive ?WQ_FLAG_EXCLUSIVE :0 ); \for (;;) { \
Signature · 1 line
1.9.1
PATCH v9 22/25 dept: Apply timeout consideration to dma fence wait
In reply to Byungchul ParkNow that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to dma fence wait.
--- drivers/dma-buf/dma-fence.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/dma-buf/dma-fence.c +2 -2
@@ -783,7 +783,7 @@ struct default_wait_cb { cb.task = current;list_add (&cb.base.node, &fence->cb_list); -sdt_might_sleep_start (NULL ); +sdt_might_sleep_start_timeout (NULL , timeout);while (!test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &fence->flags) && ret >0 ) {if (intr)__set_current_state (TASK_INTERRUPTIBLE );
@@ -887,7 +887,7 @@ struct default_wait_cb { } } -sdt_might_sleep_start (NULL ); +sdt_might_sleep_start_timeout (NULL , timeout);while (ret >0 ) {if (intr)set_current_state (TASK_INTERRUPTIBLE );
Signature · 1 line
1.9.1
PATCH v9 23/25 dept: Record the latest one out of consecutive waits of the same class
In reply to Byungchul ParkThe current code records all the waits for later use to track relation between waits and events in each context. However, since the same class is handled the same way, it'd be okay to record only one on behalf of the others if they all have the same class.
Even though it's the ideal to search the whole history buffer for that, since it'd cost too high, alternatively, let's keep the latest one at least when the same class'ed waits consecutively appear.
--- kernel/dependency/dept.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
kernel/dependency/dept.c +20 -1
@@ -1522,9 +1522,28 @@ static inline struct dept_wait_hist *new_hist(void)return wh; } +static inline struct dept_wait_hist *last_hist (void ) +{ +int pos_n =hist_pos_next (); +struct dept_wait_hist *wh_n =hist (pos_n); + +/* +* This is the first try. +*/ +if (!pos_n && !wh_n->wait) +return NULL ; + +return hist (pos_n +DEPT_MAX_WAIT_HIST -1 ); +} +static void add_hist (struct dept_wait *w,unsigned int wg,unsigned int ctxt_id) { -struct dept_wait_hist *wh =new_hist (); +struct dept_wait_hist *wh; + + wh =last_hist (); + +if (!wh || wh->wait->class != w->class || wh->ctxt_id != ctxt_id) + wh =new_hist ();if (likely (wh->wait))put_wait (wh->wait);
Signature · 1 line
1.9.1
PATCH v9 24/25 dept: Make Dept able to work with an external wgen
In reply to Byungchul ParkThere is a case where total maps for its wait/event is so large in size. For instance, struct page for PG_locked and PG_writeback is the case. The additional memory size for the maps would be 'the # of pages * sizeof(struct dept_map)' if each struct page keeps its map all the way, which might be too big to accept.
It'd be better to keep the minimum data in the case, which is timestamp called 'wgen' that Dept makes use of. So made Dept able to work with an external wgen when needed.
--- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 4 ++-- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 37 insertions(+), 15 deletions(-)
include/linux/dept.h +14 -4
@@ -487,6 +487,13 @@ struct dept_task {bool in_sched; }; +/* +* for subsystems that requires compact use of memory e.g. struct page +*/ +struct dept_ext_wgen { +unsigned int wgen; +}; +#define DEPT_TASK_INITIALIZER (t) \ { \ .wait_hist = { { .wait = NULL, } }, \
@@ -518,6 +525,7 @@ struct dept_task {extern void dept_free_range (void *start,unsigned int sz);extern void dept_map_init (struct dept_map *m,struct dept_key *k,int sub_u,const char *n);extern void dept_map_reinit (struct dept_map *m,struct dept_key *k,int sub_u,const char *n); +extern void dept_ext_wgen_init (struct dept_ext_wgen *ewg);extern void dept_map_copy (struct dept_map *to,struct dept_map *from);extern void dept_wait (struct dept_map *m,unsigned long w_f,unsigned long ip,const char *w_fn,int sub_l,long timeout);
@@ -527,8 +535,8 @@ struct dept_task {extern void dept_stage_event (struct task_struct *t,unsigned long ip);extern void dept_ecxt_enter (struct dept_map *m,unsigned long e_f,unsigned long ip,const char *c_fn,const char *e_fn,int sub_l);extern bool dept_ecxt_holding (struct dept_map *m,unsigned long e_f); -extern void dept_request_event (struct dept_map *m); -extern void dept_event (struct dept_map *m,unsigned long e_f,unsigned long ip,const char *e_fn); +extern void dept_request_event (struct dept_map *m,struct dept_ext_wgen *ewg); +extern void dept_event (struct dept_map *m,unsigned long e_f,unsigned long ip,const char *e_fn,struct dept_ext_wgen *ewg);extern void dept_ecxt_exit (struct dept_map *m,unsigned long e_f,unsigned long ip);extern void dept_sched_enter (void );extern void dept_sched_exit (void );
@@ -559,6 +567,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m)struct dept_key { };struct dept_map { };struct dept_task { }; +struct dept_ext_wgen { };#define DEPT_MAP_INITIALIZER (n, k) { }#define DEPT_TASK_INITIALIZER (t) { }
@@ -571,6 +580,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m)#define dept_free_range (s, sz) do { } while (0)#define dept_map_init (m, k, su, n) do { (void)(n); (void)(k); } while (0)#define dept_map_reinit (m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_ext_wgen_init (wg) do { } while (0)#define dept_map_copy (t, f) do { } while (0)#define dept_wait (m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0)
@@ -580,8 +590,8 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m)#define dept_stage_event (t, ip) do { } while (0)#define dept_ecxt_enter (m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0)#define dept_ecxt_holding (m, e_f) false -#define dept_request_event (m) do { } while (0) -#define dept_event (m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event (m, wg) do { } while (0) +#define dept_event (m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0)#define dept_ecxt_exit (m, e_f, ip) do { } while (0)#define dept_sched_enter () do { } while (0)#define dept_sched_exit () do { } while (0)
include/linux/dept_sdt.h +2 -2
@@ -24,7 +24,7 @@#define sdt_wait_timeout (m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0)#define sdt_wait (m) sdt_wait_timeout(m, -1L)
@@ -49,7 +49,7 @@#define sdt_might_sleep_end () dept_clean_stage()#define sdt_ecxt_enter (m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) -#define sdt_event (m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event (m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL)#define sdt_ecxt_exit (m) dept_ecxt_exit(m, 1UL, _THIS_IP_)#else /* !CONFIG_DEPT */ #define sdt_map_init (m) do { } while (0)
kernel/dependency/dept.c +21 -9
@@ -2230,6 +2230,11 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, }EXPORT_SYMBOL_GPL (dept_map_reinit); +void dept_ext_wgen_init (struct dept_ext_wgen *ewg) +{ +WRITE_ONCE (ewg->wgen,0U ); +} +void dept_map_copy (struct dept_map *to,struct dept_map *from) {if (unlikely (!dept_working ())) {
@@ -2415,7 +2420,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f,*/ static void __dept_event (struct dept_map *m,unsigned long e_f,unsigned long ip,const char *e_fn, -bool sched_map) +bool sched_map,unsigned int *wgp) {struct dept_class *c;struct dept_key *k;
@@ -2437,14 +2442,14 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, c =check_new_class (&m->map_key, k,sub_id (m, e), m->name, sched_map);if (c &&add_ecxt (m, c,0UL ,NULL , e_fn,0 )) { -do_event (m, c,READ_ONCE (m->wgen), ip); +do_event (m, c,READ_ONCE (*wgp), ip);pop_ecxt (m, c); } exit:/* * Keep the map diabled until the next sleep. */ -WRITE_ONCE (m->wgen,0U ); +WRITE_ONCE (*wgp,0U ); } void dept_wait(struct dept_map *m, unsigned long w_f,
@@ -2654,7 +2659,7 @@ void dept_stage_event(struct task_struct *t, unsigned long ip)if (!m.keys) goto exit; -__dept_event (&m,1UL , ip,"try_to_wake_up" , sched_map); +__dept_event (&m,1UL , ip,"try_to_wake_up" , sched_map, &m.wgen); exit:dept_exit (flags); }
@@ -2833,10 +2838,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) }EXPORT_SYMBOL_GPL (dept_ecxt_holding); -void dept_request_event (struct dept_map *m) +void dept_request_event (struct dept_map *m,struct dept_ext_wgen *ewg) {unsigned long flags;unsigned int wg; +unsigned int *wgp;if (unlikely (!dept_working ()))return ;
@@ -2849,32 +2855,38 @@ void dept_request_event(struct dept_map *m)*/ flags =dept_enter_recursive (); + wgp = ewg ? &ewg->wgen : &m->wgen; +/* * Avoid zero wgen. */ wg =atomic_inc_return (&wgen) ?:atomic_inc_return (&wgen); -WRITE_ONCE (m->wgen, wg); +WRITE_ONCE (*wgp, wg);dept_exit_recursive (flags); }EXPORT_SYMBOL_GPL (dept_request_event);void dept_event (struct dept_map *m,unsigned long e_f, -unsigned long ip,const char *e_fn) +unsigned long ip,const char *e_fn, +struct dept_ext_wgen *ewg) {struct dept_task *dt =dept_task ();unsigned long flags; +unsigned int *wgp;if (unlikely (!dept_working ()))return ; + wgp = ewg ? &ewg->wgen : &m->wgen; +if (dt->recursive) {/* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ -WRITE_ONCE (m->wgen,0U ); +WRITE_ONCE (*wgp,0U );return ; }
@@ -2883,7 +2895,7 @@ void dept_event(struct dept_map *m, unsigned long e_f, flags =dept_enter (); -__dept_event (m, e_f, ip, e_fn, false); +__dept_event (m, e_f, ip, e_fn, false, wgp);dept_exit (flags); }
Signature · 1 line
1.9.1
PATCH v9 25/25 dept: Track the potential waits of PG_{locked,writeback}
In reply to Byungchul ParkCurrently, Dept only tracks the real waits of PG_{locked,writeback} that actually happened having gone through __schedule() to avoid false positives. However, it ends in limited capacity for deadlock detection, because anyway there might be still way more potential dependencies by the waits that have yet to happen but may happen in the future so as to cause a deadlock.
So let Dept assume that when PG_{locked,writeback} bit gets cleared, there might be waits on the bit to be woken up.
Even though false positives may increase with the aggressive tracking, it's worth doing it because it's going to be useful in practice. See the following link for instance:
https://lore.kernel.org/lkml/[email protected]/
--- include/linux/mm_types.h | 3 ++ include/linux/page-flags.h | 112 ++++++++++++++++++++++++++++++++++++++++----- include/linux/pagemap.h | 7 ++- mm/filemap.c | 11 ++++- mm/page_alloc.c | 3 ++ 5 files changed, 121 insertions(+), 15 deletions(-)
include/linux/mm_types.h +3 -0
@@ -19,6 +19,7 @@#include <linux/workqueue.h> #include <linux/seqlock.h> #include <linux/percpu_counter.h> +#include <linux/dept.h> #include <asm/mmu.h>
@@ -252,6 +253,8 @@ struct page {#ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS int _last_cpupid;#endif +struct dept_ext_wgen PG_locked_wgen; +struct dept_ext_wgen PG_writeback_wgen; } _struct_page_alignment;/*
include/linux/page-flags.h +100 -12
150 lines not on this page · raw message
include/linux/pagemap.h +6 -1
@@ -898,7 +898,12 @@ bool __folio_lock_or_retry(struct folio *folio, struct mm_struct *mm,*/ static inline bool folio_trylock (struct folio *folio) { -return likely (!test_and_set_bit_lock (PG_locked,folio_flags (folio,0 ))); +bool ret = !test_and_set_bit_lock (PG_locked,folio_flags (folio,0 )); + +if (ret) +dept_page_set_bit (&folio->page, PG_locked); + +return likely (ret); }/*
mm/filemap.c +9 -2
@@ -1097,6 +1097,7 @@ static int wake_page_function(wait_queue_entry_t *wait, unsigned mode, int sync,if (flags &WQ_FLAG_CUSTOM ) {if (test_and_set_bit (key->bit_nr, &key->folio->flags))return -1 ; +dept_page_set_bit (&key->folio->page, key->bit_nr); flags |=WQ_FLAG_DONE ; } }
@@ -1206,6 +1207,7 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr,if (wait->flags &WQ_FLAG_EXCLUSIVE ) {if (test_and_set_bit (bit_nr, &folio->flags))return false; +dept_page_set_bit (&folio->page, bit_nr); }else if (test_bit (bit_nr, &folio->flags))return false;
@@ -1216,8 +1218,10 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr,/* How many times do we accept lock stealing from under a waiter? */ int sysctl_page_lock_unfairness =5 ; -static struct dept_map __maybe_unused PG_locked_map =DEPT_MAP_INITIALIZER (PG_locked_map,NULL ); -static struct dept_map __maybe_unused PG_writeback_map =DEPT_MAP_INITIALIZER (PG_writeback_map,NULL ); +struct dept_map __maybe_unused PG_locked_map =DEPT_MAP_INITIALIZER (PG_locked_map,NULL ); +struct dept_map __maybe_unused PG_writeback_map =DEPT_MAP_INITIALIZER (PG_writeback_map,NULL ); +EXPORT_SYMBOL (PG_locked_map); +EXPORT_SYMBOL (PG_writeback_map);static inline int folio_wait_bit_common (struct folio *folio,int bit_nr,int state,enum behavior behavior)
@@ -1230,6 +1234,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr,unsigned long pflags;bool in_thrashing; +dept_page_wait_on_bit (&folio->page, bit_nr);if (bit_nr == PG_locked)sdt_might_sleep_start (&PG_locked_map);else if (bit_nr == PG_writeback)
@@ -1327,6 +1332,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, wait->flags |=WQ_FLAG_DONE ;break ; } +dept_page_set_bit (&folio->page, bit_nr);/* * If a signal happened, this 'finish_wait()' may remove the last
@@ -1534,6 +1540,7 @@ void folio_unlock(struct folio *folio)BUILD_BUG_ON (PG_waiters !=7 );BUILD_BUG_ON (PG_locked >7 );VM_BUG_ON_FOLIO (!folio_test_locked (folio), folio); +dept_page_clear_bit (&folio->page, PG_locked);if (clear_bit_unlock_is_negative_byte (PG_locked,folio_flags (folio,0 )))folio_wake_bit (folio, PG_locked); }
mm/page_alloc.c +3 -0
@@ -76,6 +76,7 @@#include <linux/khugepaged.h> #include <linux/buffer_head.h> #include <linux/delayacct.h> +#include <linux/dept.h> #include <asm/sections.h> #include <asm/tlbflush.h> #include <asm/div64.h>
@@ -1626,6 +1627,8 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn,page_mapcount_reset (page);page_cpupid_reset_last (page);page_kasan_tag_reset (page); +dept_ext_wgen_init (&page->PG_locked_wgen); +dept_ext_wgen_init (&page->PG_writeback_wgen);INIT_LIST_HEAD (&page->lru);#ifdef WANT_PAGE_VIRTUAL
Signature · 1 line
1.9.1