btrfs: trace: Trace events for btrfs tree locking
Subsystems (2)
v3 cover btrfs: trace: Trace events for btrfs tree locking
This branch can be fetched from github:
https://github.com/adam900710/linux/tree/trace_locking
Which is based on v5.1-rc4 tag.
This patchset will introduce the following trace events for corresponding functions:
btrfs_tree_lock (sleepable)
btrfs_tree_unlock
btrfs_tree_read_lock (sleepable)
btrfs_tree_read_unlock
btrfs_tree_read_unlock_blocking btrfs_set_lock_blocking_read
btrfs_set_lock_blocking_write btrfs_clear_lock_blocking_read btrfs_clear_lock_blocking_write
btrfs_try_tree_read_lock
btrfs_try_tree_write_lock
btrfs_tree_read_lock_atomic
The two sleepable ones will have extra info like start_ns, end_ns, diff_ns.
Those two can be specially useful for tree lock wait time. There is tool to utilize those two events already:
https://github.com/adam900710/btrfs-profiler/blob/master/tree_lock_wait.py
The non-sleepable can be useful for possible user space based lock leakage/dead lock detector.
Changelog:
v2:
- Hide all ktime_get_ns() call out of trace events
So there will be no overhead if trace events are not enabled.
v3:
- More output for human to take a glance
- More trace events for later bcc usage
Like lock leakage and deadlock detection.
Qu Wenruo (2):
btrfs: trace: Introduce trace events for sleepable tree lock
btrfs: trace: Introduce trace events for all btrfs tree locking events
fs/btrfs/locking.c | 22 +++++++++ include/trace/events/btrfs.h | 86 ++++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+)
Signature · 1 line
2.21.0
PATCH v3 1/2 btrfs: trace: Introduce trace events for sleepable tree lock
There are two tree lock events which can sleep:
- btrfs_tree_read_lock()
- btrfs_tree_lock()
Sometimes we may need to look into the concurrency picture of the fs. For that case, we need the execution time of above two functions and the owner of @eb.
Here we introduce a trace events for user space tools like bcc, to get the execution time of above two functions, and get detailed owner info where eBPF code can't.
All the overhead is hidden behind the trace events, so if events are not enabled, there is no overhead.
These trace events also output bytenr and generation, allow them to be pared with unlock events to pin down deadlock.
--- fs/btrfs/locking.c | 12 ++++++++++ include/trace/events/btrfs.h | 44 ++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+)
fs/btrfs/locking.c +12 -0
@@ -90,6 +90,10 @@ void btrfs_clear_lock_blocking_write(struct extent_buffer *eb)*/ void btrfs_tree_read_lock (struct extent_buffer *eb) { +u64 start_ns =0 ; + +if (trace_btrfs_tree_read_lock_enabled ()) + start_ns =ktime_get_ns (); again:BUG_ON (!atomic_read (&eb->blocking_writers) && current->pid == eb->lock_owner);
@@ -106,6 +110,7 @@ void btrfs_tree_read_lock(struct extent_buffer *eb)BUG_ON (eb->lock_nested); eb->lock_nested =1 ;read_unlock (&eb->lock); +trace_btrfs_tree_read_lock (eb, start_ns);return ; }if (atomic_read (&eb->blocking_writers)) {
@@ -116,6 +121,7 @@ void btrfs_tree_read_lock(struct extent_buffer *eb) }atomic_inc (&eb->read_locks);atomic_inc (&eb->spinning_readers); +trace_btrfs_tree_read_lock (eb, start_ns); }/*
@@ -232,6 +238,11 @@ void btrfs_tree_read_unlock_blocking(struct extent_buffer *eb)*/ void btrfs_tree_lock (struct extent_buffer *eb) { +u64 start_ns =0 ; + +if (trace_btrfs_tree_lock_enabled ()) + start_ns =ktime_get_ns (); +WARN_ON (eb->lock_owner == current->pid); again:wait_event (eb->read_lock_wq,atomic_read (&eb->blocking_readers) ==0 );
@@ -246,6 +257,7 @@ void btrfs_tree_lock(struct extent_buffer *eb)atomic_inc (&eb->spinning_writers);atomic_inc (&eb->write_locks); eb->lock_owner = current->pid; +trace_btrfs_tree_lock (eb, start_ns); }/*
include/trace/events/btrfs.h +44 -0
50 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH v3 2/2 btrfs: trace: Introduce trace events for all btrfs tree locking events
In reply to Qu WenruoUnlike btrfs_tree_lock() and btrfs_tree_read_lock(), the remaining functions in locking.c will not sleep, thus doesn't make much sense to record their execution time.
Those events are introduced mainly for user space tool to audit and detect lock leakage or dead lock.
--- fs/btrfs/locking.c | 10 +++++++++ include/trace/events/btrfs.h | 42 ++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+)
fs/btrfs/locking.c +10 -0
@@ -16,6 +16,7 @@ static void btrfs_assert_tree_read_locked(struct extent_buffer *eb);void btrfs_set_lock_blocking_read (struct extent_buffer *eb) { +trace_btrfs_set_lock_blocking_read (eb);/* * No lock is required. The lock owner may change if we have a read * lock, but it won't change to or away from us. If we have the write
@@ -32,6 +33,7 @@ void btrfs_set_lock_blocking_read(struct extent_buffer *eb)void btrfs_set_lock_blocking_write (struct extent_buffer *eb) { +trace_btrfs_set_lock_blocking_write (eb);/* * No lock is required. The lock owner may change if we have a read * lock, but it won't change to or away from us. If we have the write
@@ -50,6 +52,7 @@ void btrfs_set_lock_blocking_write(struct extent_buffer *eb)void btrfs_clear_lock_blocking_read (struct extent_buffer *eb) { +trace_btrfs_clear_lock_blocking_read (eb);/* * No lock is required. The lock owner may change if we have a read * lock, but it won't change to or away from us. If we have the write
@@ -67,6 +70,7 @@ void btrfs_clear_lock_blocking_read(struct extent_buffer *eb)void btrfs_clear_lock_blocking_write (struct extent_buffer *eb) { +trace_btrfs_clear_lock_blocking_write (eb);/* * no lock is required. The lock owner may change if * we have a read lock, but it won't change to or away
@@ -141,6 +145,7 @@ int btrfs_tree_read_lock_atomic(struct extent_buffer *eb) }atomic_inc (&eb->read_locks);atomic_inc (&eb->spinning_readers); +trace_btrfs_tree_read_lock_atomic (eb);return 1 ; }
@@ -162,6 +167,7 @@ int btrfs_try_tree_read_lock(struct extent_buffer *eb) }atomic_inc (&eb->read_locks);atomic_inc (&eb->spinning_readers); +trace_btrfs_try_tree_read_lock (eb);return 1 ; }
@@ -184,6 +190,7 @@ int btrfs_try_tree_write_lock(struct extent_buffer *eb)atomic_inc (&eb->write_locks);atomic_inc (&eb->spinning_writers); eb->lock_owner = current->pid; +trace_btrfs_try_tree_write_lock (eb);return 1 ; }
@@ -192,6 +199,7 @@ int btrfs_try_tree_write_lock(struct extent_buffer *eb)*/ void btrfs_tree_read_unlock (struct extent_buffer *eb) { +trace_btrfs_tree_read_unlock (eb);/* * if we're nested, we have the write lock. No new locking * is needed as long as we are the lock owner.
@@ -214,6 +222,7 @@ void btrfs_tree_read_unlock(struct extent_buffer *eb)*/ void btrfs_tree_read_unlock_blocking (struct extent_buffer *eb) { +trace_btrfs_tree_read_unlock_blocking (eb);/* * if we're nested, we have the write lock. No new locking * is needed as long as we are the lock owner.
@@ -270,6 +279,7 @@ void btrfs_tree_unlock(struct extent_buffer *eb)BUG_ON (blockers >1 );btrfs_assert_tree_locked (eb); +trace_btrfs_tree_unlock (eb); eb->lock_owner =0 ;atomic_dec (&eb->write_locks);
include/trace/events/btrfs.h +42 -0
48 lines not on this page · raw message
Signature · 1 line
2.21.0
Quoted 33 lines · Qu Wenruo
This branch can be fetched from github:
https://github.com/adam900710/linux/tree/trace_locking
Which is based on v5.1-rc4 tag.This patchset will introduce the following trace events for corresponding functions:
btrfs_tree_lock(sleepable)
btrfs_tree_unlock
btrfs_tree_read_lock(sleepable)
btrfs_tree_read_unlock
btrfs_tree_read_unlock_blockingbtrfs_set_lock_blocking_read
btrfs_set_lock_blocking_writebtrfs_clear_lock_blocking_readbtrfs_clear_lock_blocking_write
btrfs_try_tree_read_lock
btrfs_try_tree_write_lock
btrfs_tree_read_lock_atomicThe two sleepable ones will have extra info like
start_ns,end_ns,diff_ns.
Those two can be specially useful for tree lock wait time. There is tool to utilize those two events already:
https://github.com/adam900710/btrfs-profiler/blob/master/tree_lock_wait.pyThe non-sleepable can be useful for possible user space based lock leakage/dead lock detector.
Changelog:
v2:
- Hide allktime_get_ns()call out of trace events
So there will be no overhead if trace events are not enabled.
Quoted 4 lines
v3:
- More output for human to take a glance
- More trace events for later bcc usage
Like lock leakage and deadlock detection.
V3 looks good to me, thanks. The enhanced output of the sleepable lock is still sane.