Lock dependency based tree report in perf lock
Hi,
Looking at the report layout we have with perf lock, it gives a cool overview of per lock waittime, acquire time, by avg/max, that's cool.
Now to complete the overview through another dimension we could have another mode on top of a lock dependency based tree:
-- 125 ms -- lock1
|
-- 100 ms -- lock2
| |
| -- 99 ms -- lock 3
| |
| -- 16 us -- lock 4
| |
| ----- [...]
|
-- 25 ms -- lock 5Having such view can tell you which lock is delaying another one. We could have this view by average or maximum of acquired time (waittime here would be pointless).
And we can also report the outer locking noise time for these reports.
Taking the above example:
-- 100 ms/84 us -- lock2
|
-- 99 ms -- lock 3
|
-- 16 us -- lock 4
|
----- [...]100 ms is the time lock2 has been acquired. Lock 3 has been acquired 99 ms and lock 4 during 16 us, which means the outer locking noise (the code executed outside children locks) is of 84 us.
This noise can give a good rough glance of the influence childs locks can have on a parent. This is somehow the child weight, this all can even be expressed using percentages, we could even reproduce the absolute/relative percentage we currently have with the callchains in perf report (-g graph for absolute, -g fractal for relative).
Or we can have a ghost child for each locks that represent the outer child locks noise:
-- parent percentage -- lock2
|
-- 99 % -- lock 3
|
-- 0.84 % -- noise // could have its own color for quick distinguishing
|
-- 0.16 % -- lock 4
|
----- [...]May be just one tricky thing.
Say we have lock3 and lock4 that depend on lock2, it doesn't always means lock4 will always be taken after lock3, it doesn't even mean lock4 will ever be taken after lock3.
So we shouldn't have one branch per dependency but one branch per practical scenario encountered.
So imagine we have the following dependency:
lock2
|
--- lock3
|
--- lock4
|
--- lock5
But we can actually have only the following scenarios:
lock2
|
-- lock3
|
-- lock4
lock2
|
-- lock4
lock2
|
-- lock5
Then we need three branches:
-- lock2 percentage -- lock2
|
-- 60 %
| |
| -- 90 % -- lock 3
| |
| -- 9 % -- noise
| |
| -- 1 % -- lock 4
|
-- 20 %
| |
| -- 99 % -- lock4
| |
| -- 1 % -- noise
|
-- 20 %
|
-- 14 % -- lock5
|
-- 86 % -- noiseHopefully we could have such expandable tree using ncurses, but we can probably start like callchains in perf report.
There might also be a problem with the accuracy. The max time lock2 is acquired won't always match the path where lock3 and lock4 had their max time too, but this should give, most of the time, a view close to the reality, especially once we enter non-sleepable lock branches.
Anyway, that's just an idea, not trivial I must admit.
Quoted 3 lines · Frederic Weisbecker
Anyway, that's just an idea, not trivial I must admit.
lockdep actually collects all this information, so writing it out isn't too hard.
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Frederic Weisbecker
Anyway, that's just an idea, not trivial I must admit.
lockdep actually collects all this information, so writing it out isn't too hard.
Lockdep collects the theorical dependencies but not the practical scenarios.
Say B and C depend on A, you'll get:
A
/ \
B C
But nothing can tell you that if A is taken, B and C will always be taken. You may have different scenarios based on this dependency, which is not something that lockdep logs, right?
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Frederic Weisbecker
Anyway, that's just an idea, not trivial I must admit.
lockdep actually collects all this information, so writing it out isn't too hard.
Hmm, I'm discovering the /proc/lock_stat file this evening, did not know it exist :)
Still, a tree representation can bring another dimension.
Quoted 16 lines · Frederic Weisbecker
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Frederic Weisbecker
Anyway, that's just an idea, not trivial I must admit.
lockdep actually collects all this information, so writing it out isn't too hard.
Lockdep collects the theorical dependencies but not the practical scenarios.
Say B and C depend on A, you'll get:
Quoted 7 lines
A
/ \
B CBut nothing can tell you that if A is taken, B and C will always be taken. You may have different scenarios based on this dependency, which is not something that lockdep logs, right?
Right. But we keep track of the full held lock stack, which is what was requested.
Quoted 6 lines · Frederic Weisbecker
Quoted 5 lines · Peter Zijlstra
Quoted 3 lines · Frederic Weisbecker
Anyway, that's just an idea, not trivial I must admit.
Quoted 8 lines
Quoted 2 lines
lockdep actually collects all this information, so writing it out isn't too hard.
Hmm, I'm discovering the
/proc/lock_statfile this evening, did not know it exist :)Still, a tree representation can bring another dimension.
current->held_locks[i]->instance, i < current->lock_depth
Quoted 21 lines · Peter Zijlstra
Quoted 20 lines · Frederic Weisbecker
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Frederic Weisbecker
Anyway, that's just an idea, not trivial I must admit.
lockdep actually collects all this information, so writing it out isn't too hard.
Lockdep collects the theorical dependencies but not the practical scenarios.
Say B and C depend on A, you'll get:
A
/ \
B C
Quoted 6 lines
Quoted 3 lines
But nothing can tell you that if A is taken, B and C will always be taken. You may have different scenarios based on this dependency, which is not something that lockdep logs, right?
Right. But we keep track of the full held lock stack, which is what was requested.
Ah, I see what you mean. Yeah we have that, but it is a runtime check feature, and the representation is linear. We should not add further tracepoint based on this to get the possible scenarios of locking, we should rather deduce them from the current tracepoints we have.
Would be a bad idea to add even more overhead.