kretprobe: Prevent triggering kretprobe from within kprobe_flush_task
Subsystems (3)
RFC 1/1 kretprobe: Prevent triggering kretprobe from within kprobe_flush_task
hi,
Ziqian reported lockup when adding retprobe on _raw_spin_lock_irqsave. My test was also able to trigger lockdep output:
============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_hash_lock+0x52/0xa0
but task is already holding lock:
ffffffff9a491a18 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_trampoline+0x0/0x50
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
---- lock(&(kretprobe_table_locks[i].lock));
lock(&(kretprobe_table_locks[i].lock));
*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0: ffffffff9a491a18 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_trampoline+0x0/0x50
stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any _raw_spin_lock_irqsave probe in there.
The problem is in outside kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsave
where _raw_spin_lock_irqsave triggers the kretprobe and installs kretprobe_trampoline handler on _raw_spin_lock_irqsave return.
The kretprobe_trampoline handler is then executed with already locked kretprobe_table_locks, and first thing it does is to lock kretprobe_table_locks ;-) the whole lockup path like:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed
---> kretprobe_table_locks locked
kretprobe_trampoline
trampoline_handler
kretprobe_hash_lock(current, &head, &flags); <--- deadlockThe change below sets current_kprobe in kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.
I'm not sure this is the best fix, any ideas are welcome ;-)
thanks,
jirka
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)
kernel/kprobes.c +6 -0
@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated
@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);kretprobe_table_lock (hash, &flags);hlist_for_each_entry_safe (ri, tmp, head, hlist) {if (ri->task == tk)recycle_rp_inst (ri, &empty_rp); }kretprobe_table_unlock (hash, &flags); +__this_cpu_write (current_kprobe,NULL );hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kfree (ri);
Signature · 1 line
2.25.2
Hi Jiri,
Quoted 82 lines · Jiri Olsa
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave. My test was also able to trigger lockdep output:============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlock
Quoted 3 lines
The change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.
Good analysis!
Quoted 2 lines
I'm not sure this is the best fix, any ideas are welcome ;-)
I think this is a good way to address this issue.
Quoted 14 lines
thanks,
jirka--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +0
-0
kernel/kprobes.c __releases(hlist_lock) · line 1242
Quoted 7 lines
kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +
Perhaps a more meaningful name, say, kprobe_flush_task_protect ?
Quoted 5 lines
kernel/kprobes.c +0
-0/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated @@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp);
kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1270
Quoted 10 lines
kernel/kprobes.c +2
-0hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);kretprobe_table_lock (hash, &flags);hlist_for_each_entry_safe (ri, tmp, head, hlist) {if (ri->task == tk)recycle_rp_inst (ri, &empty_rp); }kretprobe_table_unlock (hash, &flags); +__this_cpu_write (current_kprobe,NULL );
I would move this to the end of the function to also cover the below code. kprobe_flush_task() is marked NOKPROBE, so it is good to prevent probe handling in the below code too.
kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1273
Quoted 3 lines
kernel/kprobes.c +0
-0hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kfree (ri);
- Naveen
Masami Hiramatsu
maintainerHi Jiri,
Quoted 2 lines · On Wed, 8 Apr 2020 18:46:41 +0200 Jiri Olsa <[email protected]> wrote:
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave.
Hmm, kprobe is lockless, but kretprobe involves spinlock. Thus, eventually, I will blacklist the _raw_spin_lock_irqsave() for kretprobe.
If you need to trace spinlock return, please consider to putting kprobe at "ret" instruction.
Quoted 57 lines
My test was also able to trigger lockdep output:
============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
Quoted 7 lines
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.
Hmm, OK. In this case, I think we should mark this process is going to die and never try to kretprobe on it.
Quoted 16 lines
The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlock
Quoted 5 lines
The change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.I'm not sure this is the best fix, any ideas are welcome ;-)
Hmm, this is a bit tricky to fix this issue. Of course, temporary disable kprobes (and kretprobe) on an area by filling current_kprobe might be a good idea, but it also involves other kprobes.
How about let kretprobe skip the task which state == TASK_DEAD ?
kernel/kprobes.c +5 -2
@@ -1874,9 +1874,12 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs)* To avoid deadlocks, prohibit return probing in NMI contexts, * just skip the probe and increase the (inexact) 'nmissed' * statistical counter, so that the user is informed that -* something happened: +* something happened. +* Also, if the current task is dead, we will already in the process +* to reclaim kretprobe instances from hash list. To avoid memory +* leak, skip to run the kretprobe on such task. */ -if (unlikely (in_nmi ())) { +if (unlikely (in_nmi ()) || current->state ==TASK_DEAD ) { rp->nmissed++;return 0 ; }
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 11 lines · Masami Hiramatsu
Hi Jiri,
Quoted 2 lines · On Wed, 8 Apr 2020 18:46:41 +0200 Jiri Olsa <[email protected]> wrote:
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave.Hmm, kprobe is lockless, but kretprobe involves spinlock. Thus, eventually, I will blacklist the
_raw_spin_lock_irqsave()for kretprobe.
I thought of blacklisting, but we were using that kretprobe in a bcc script.. it's not overloaded by using bpf trampolines, but still might be useful
SNIP
Quoted 36 lines
Quoted 5 lines
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:Quoted 7 lines
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.Hmm, OK. In this case, I think we should mark this process is going to die and never try to kretprobe on it.
Quoted 20 lines
The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockThe change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.
Quoted 7 lines
Quoted 1 line
I'm not sure this is the best fix, any ideas are welcome ;-)
Hmm, this is a bit tricky to fix this issue. Of course, temporary disable kprobes (and kretprobe) on an area by filling
current_kprobemight be a good idea, but it also involves other kprobes.How about let kretprobe skip the task which state ==
TASK_DEAD?
hum, isn't that considerable amount of paths (with state == TASK_DEAD) that we would kill kprobes for? someone might want to trace it
thanks,
jirka
Hi Masami,
Quoted 11 lines · Masami Hiramatsu
Hi Jiri,
Quoted 2 lines · On Wed, 8 Apr 2020 18:46:41 +0200 Jiri Olsa <[email protected]> wrote:
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave.Hmm, kprobe is lockless, but kretprobe involves spinlock. Thus, eventually, I will blacklist the
_raw_spin_lock_irqsave()for kretprobe.
As far as I can see, this is the only place where probing _raw_spin_lock_irqsave() is an issue. Should we blacklist all users for this case alone?
Quoted 91 lines
If you need to trace spinlock return, please consider to putting kprobe at "ret" instruction.
Quoted 57 lines
My test was also able to trigger lockdep output:
============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:Quoted 7 lines
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.Hmm, OK. In this case, I think we should mark this process is going to die and never try to kretprobe on it.
Quoted 20 lines
The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockThe change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.
Quoted 5 lines
Quoted 1 line
I'm not sure this is the best fix, any ideas are welcome ;-)
Hmm, this is a bit tricky to fix this issue. Of course, temporary disable kprobes (and kretprobe) on an area by filling
current_kprobemight be a good idea, but it also involves other kprobes.
Not sure how you mean that. Jiri's RFC patch would be disabling k[ret]probes within kprobe_flush_task(), which is only ever invoked from finish_task_switch(). I only see calls to spin locks and kfree() from here. Besides, kprobe_flush_task() itself is NOKPROBE, so we would ideally want to not trace/probe other functions it calls.
Quoted 9 lines
How about let kretprobe skip the task which state ==
TASK_DEAD?kernel/kprobes.c +0
-0@@ -1874,9 +1874,12 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs)* To avoid deadlocks, prohibit return probing in NMI contexts,
kernel/kprobes.c static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs) · line 1882
Quoted 10 lines
kernel/kprobes.c +5
-2* just skip the probe and increase the (inexact) 'nmissed' * statistical counter, so that the user is informed that -* something happened: +* something happened. +* Also, if the current task is dead, we will already in the process +* to reclaim kretprobe instances from hash list. To avoid memory +* leak, skip to run the kretprobe on such task. */ -if (unlikely (in_nmi ())) { +if (unlikely (in_nmi ()) || current->state ==TASK_DEAD ) {
I'm wondering if this actually works. kprobe_flush_task() seems to be called from finish_task_switch(), after the task switch is complete. So, current task won't actually be dead here.
- Naveen
Masami Hiramatsu
maintainerQuoted 57 lines · On Thu, 9 Apr 2020 14:52:13 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 11 lines · Masami Hiramatsu
Hi Jiri,
Quoted 2 lines · On Wed, 8 Apr 2020 18:46:41 +0200 Jiri Olsa <[email protected]> wrote:
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave.Hmm, kprobe is lockless, but kretprobe involves spinlock. Thus, eventually, I will blacklist the
_raw_spin_lock_irqsave()for kretprobe.I thought of blacklisting, but we were using that kretprobe in a bcc script.. it's not overloaded by using bpf trampolines, but still might be useful
SNIP
Quoted 38 lines
Quoted 5 lines
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:Quoted 7 lines
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.Hmm, OK. In this case, I think we should mark this process is going to die and never try to kretprobe on it.
Quoted 21 lines
The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockThe change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.I'm not sure this is the best fix, any ideas are welcome ;-)
Quoted 8 lines
Quoted 5 lines
Hmm, this is a bit tricky to fix this issue. Of course, temporary disable kprobes (and kretprobe) on an area by filling
current_kprobemight be a good idea, but it also involves other kprobes.How about let kretprobe skip the task which state ==
TASK_DEAD?hum, isn't that considerable amount of paths (with state ==
TASK_DEAD) that we would kill kprobes for? someone might want to trace it
OK, and I found that even after calling kprobe_flush_task(), it may be work because the task will not be switched. kretprobe instance will be reclaimed.
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerHi,
Quoted 11 lines · On Thu, 09 Apr 2020 18:46:47 +0530 "Naveen N. Rao"…
Hi Masami,
Quoted 8 lines · Masami Hiramatsu
Hi Jiri,
Quoted 2 lines · On Wed, 8 Apr 2020 18:46:41 +0200 Jiri Olsa <[email protected]> wrote:
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave.
Quoted 7 lines
Quoted 3 lines
Hmm, kprobe is lockless, but kretprobe involves spinlock. Thus, eventually, I will blacklist the
_raw_spin_lock_irqsave()for kretprobe.As far as I can see, this is the only place where probing
_raw_spin_lock_irqsave()is an issue. Should we blacklist all users for this case alone?
Hrm, right. kretprobe is different from kprobe's case.
Quoted 98 lines
Quoted 92 lines
If you need to trace spinlock return, please consider to putting kprobe at "ret" instruction.
Quoted 57 lines
My test was also able to trigger lockdep output:
============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:Quoted 7 lines
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.Hmm, OK. In this case, I think we should mark this process is going to die and never try to kretprobe on it.
Quoted 20 lines
The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockThe change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.Quoted 5 lines
Quoted 1 line
I'm not sure this is the best fix, any ideas are welcome ;-)
Hmm, this is a bit tricky to fix this issue. Of course, temporary disable kprobes (and kretprobe) on an area by filling
current_kprobemight be a good idea, but it also involves other kprobes.
Quoted 5 lines
Not sure how you mean that. Jiri's RFC patch would be disabling k[ret]probes within
kprobe_flush_task(), which is only ever invoked fromfinish_task_switch(). I only see calls to spin locks andkfree()from here. Besides,kprobe_flush_task()itself is NOKPROBE, so we would ideally want to not trace/probe other functions it calls.
Ah, good point. I forgot that has been blacklisted. OK. then I accept the Jiri's RFC.
Thank you,
Quoted 31 lines
Quoted 10 lines
How about let kretprobe skip the task which state ==
TASK_DEAD?kernel/kprobes.c +0
-0@@ -1874,9 +1874,12 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs)* To avoid deadlocks, prohibit return probing in NMI contexts, kernel/kprobes.c static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs) · line 1882
Quoted 10 lines
kernel/kprobes.c +5
-2* just skip the probe and increase the (inexact) 'nmissed' * statistical counter, so that the user is informed that -* something happened: +* something happened. +* Also, if the current task is dead, we will already in the process +* to reclaim kretprobe instances from hash list. To avoid memory +* leak, skip to run the kretprobe on such task. */ -if (unlikely (in_nmi ())) { +if (unlikely (in_nmi ()) || current->state ==TASK_DEAD ) {I'm wondering if this actually works.
kprobe_flush_task()seems to be called fromfinish_task_switch(), after the task switch is complete. So, current task won't actually be dead here.- Naveen
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerQuoted 82 lines · On Wed, 8 Apr 2020 18:46:41 +0200 Jiri Olsa <[email protected]> wrote:
hi,
Ziqian reported lockup when adding retprobe on_raw_spin_lock_irqsave. My test was also able to trigger lockdep output:============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlock
Quoted 5 lines
The change below sets
current_kprobeinkprobe_flush_task, so the probe recursion protection check is hit and the probe is never set. It seems to fix the deadlock.I'm not sure this is the best fix, any ideas are welcome ;-)
OK, I just have 1 comment. :)
Quoted 25 lines
thanks,
jirka--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated
kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1263
Quoted 4 lines
kernel/kprobes.c +1
-0@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);
Can you also set the kcb->kprobe_state = KPROBE_HIT_ACTIVE?
BTW, we may be better to introduce a common kprobe_reject_section_start() and kprobe_reject_section_end() so that the user don't need to prepare dummy kprobes.
Thank you,
Quoted 13 lines
kernel/kprobes.c +1
-0kretprobe_table_lock (hash, &flags);hlist_for_each_entry_safe (ri, tmp, head, hlist) {if (ri->task == tk)recycle_rp_inst (ri, &empty_rp); }kretprobe_table_unlock (hash, &flags); +__this_cpu_write (current_kprobe,NULL );hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kfree (ri);Signature · 1 line
2.25.2
Signature · 1 line
Masami Hiramatsu <[email protected]>
On Thu, Apr 09, 2020 at 02:32:21PM +0530, Naveen N. Rao wrote:
SNIP
Quoted 4 lines
Quoted 4 lines
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)
Quoted 13 lines
kernel/kprobes.c __releases(hlist_lock) · line 1241
Quoted 11 lines
kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +Perhaps a more meaningful name, say,
kprobe_flush_task_protect?
ok, will rename together with Masami's changes
Quoted 12 lines
Quoted 12 lines
kernel/kprobes.c +1
-0/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated @@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);kretprobe_table_lock (hash, &flags);hlist_for_each_entry_safe (ri, tmp, head, hlist) {if (ri->task == tk)
Quoted 8 lines
kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1270
Quoted 4 lines
kernel/kprobes.c +1
-0recycle_rp_inst (ri, &empty_rp); }kretprobe_table_unlock (hash, &flags); +__this_cpu_write (current_kprobe,NULL );I would move this to the end of the function to also cover the below code.
kprobe_flush_task()is marked NOKPROBE, so it is good to prevent probe handling in the below code too.
ok, will do
thanks,
jirka
Quoted 8 lines
kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1273
Quoted 4 lines
kernel/kprobes.c +0
-0hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kfree (ri);- Naveen
On Thu, Apr 09, 2020 at 11:41:01PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 27 lines
Quoted 20 lines
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1263
Quoted 4 lines
kernel/kprobes.c +1
-0@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);Can you also set the
kcb->kprobe_state=KPROBE_HIT_ACTIVE?
Quoted 3 lines
BTW, we may be better to introduce a common
kprobe_reject_section_start()andkprobe_reject_section_end()so that the user don't need to prepare dummy kprobes.
sure, will do
thank you both for review
jirka
Quoted 21 lines
Thank you,
Quoted 13 lines
kernel/kprobes.c +1
-0kretprobe_table_lock (hash, &flags);hlist_for_each_entry_safe (ri, tmp, head, hlist) {if (ri->task == tk)recycle_rp_inst (ri, &empty_rp); }kretprobe_table_unlock (hash, &flags); +__this_cpu_write (current_kprobe,NULL );hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kfree (ri);Signature · 1 line
2.25.2Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 31 lines · Jiri Olsa
On Thu, Apr 09, 2020 at 11:41:01PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 27 lines
Quoted 20 lines
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1263
Quoted 4 lines
kernel/kprobes.c +1
-0@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);Can you also set the
kcb->kprobe_state=KPROBE_HIT_ACTIVE?
Quoted 7 lines
Quoted 3 lines
BTW, we may be better to introduce a common
kprobe_reject_section_start()andkprobe_reject_section_end()so that the user don't need to prepare dummy kprobes.sure, will do
thank you both for review
ok, found out it's actually arch code.. would you guys be ok with something like below?
jirka
---
arch/x86/kernel/kprobes/core.c +24 -13
@@ -757,12 +757,33 @@ static struct kprobe kretprobe_kprobe = { .addr = (void *)kretprobe_trampoline, }; +void arch_kprobe_reject_section_start (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable (); + +/* +* Set a dummy kprobe for avoiding kretprobe recursion. +* Since kretprobe never run in kprobe handler, kprobe must not +* be running behind this point. +*/ +__this_cpu_write (current_kprobe, &kretprobe_kprobe); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +} + +void arch_kprobe_reject_section_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;
@@ -772,16 +793,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); - -/* -* Set a dummy kprobe for avoiding kretprobe recursion. -* Since kretprobe never run in kprobe handler, kprobe must not -* be running at this point. -*/ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +arch_kprobe_reject_section_start ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);
@@ -873,8 +885,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +arch_kprobe_reject_section_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
kernel/kprobes.c +12 -0
@@ -1236,6 +1236,14 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +void __weak arch_kprobe_reject_section_start (void ) +{ +} + +void __weak arch_kprobe_reject_section_end (void ) +{ +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated
@@ -1253,6 +1261,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +arch_kprobe_reject_section_start (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];
@@ -1266,6 +1276,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +arch_kprobe_reject_section_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);
Masami Hiramatsu
maintainerHi Jiri,
Quoted 36 lines · On Thu, 9 Apr 2020 22:13:36 +0200 Jiri Olsa <[email protected]> wrote:
Quoted 35 lines · Jiri Olsa
On Thu, Apr 09, 2020 at 11:41:01PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 30 lines
Quoted 20 lines
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1263
Quoted 4 lines
kernel/kprobes.c +1
-0@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);Can you also set the
kcb->kprobe_state=KPROBE_HIT_ACTIVE?BTW, we may be better to introduce a common
kprobe_reject_section_start()andkprobe_reject_section_end()so that the user don't need to prepare dummy kprobes.
Quoted 5 lines
Quoted 3 lines
sure, will do
thank you both for review
ok, found out it's actually arch code.. would you guys be ok with something like below?
Thanks for update!
Quoted 18 lines
jirka
---
arch/x86/kernel/kprobes/core.c +5
-0@@ -757,12 +757,33 @@ static struct kprobe kretprobe_kprobe = { .addr = (void *)kretprobe_trampoline, }; +void arch_kprobe_reject_section_start (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable ();
arch/x86/kernel/kprobes/core.c static struct kprobe kretprobe_kprobe = { · line 774
Quoted 10 lines
arch/x86/kernel/kprobes/core.c +10
-0+ +/* +* Set a dummy kprobe for avoiding kretprobe recursion. +* Since kretprobe never run in kprobe handler, kprobe must not +* be running behind this point. +*/ +__this_cpu_write (current_kprobe, &kretprobe_kprobe); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +}
Yeah, the code seems good to me.
BTW, I rather like make it arch independent function so that other arch can use it. In this case, the dummy kprobe's addr should be somewhere obviously blacklisted (but it must be accessible.) I think get_kprobe() will be a candidate.
And (sorry about changing my mind), the naming, I think kprobe_busy_begin() and kprobe_busy_end() will be better because it doesn't reject registering kprobes, instead, just make it busy :)
Thank you,
Quoted 82 lines
arch/x86/kernel/kprobes/core.c +9
-13+ +void arch_kprobe_reject_section_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;@@ -772,16 +793,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); - -/* -* Set a dummy kprobe for avoiding kretprobe recursion. -* Since kretprobe never run in kprobe handler, kprobe must not -* be running at this point. -*/ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +arch_kprobe_reject_section_start ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);@@ -873,8 +885,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +arch_kprobe_reject_section_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kernel/kprobes.c +12
-0@@ -1236,6 +1236,14 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +void __weak arch_kprobe_reject_section_start (void ) +{ +} + +void __weak arch_kprobe_reject_section_end (void ) +{ +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated @@ -1253,6 +1261,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +arch_kprobe_reject_section_start (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];@@ -1266,6 +1276,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +arch_kprobe_reject_section_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 36 lines · Jiri Olsa
Quoted 35 lines · Jiri Olsa
On Thu, Apr 09, 2020 at 11:41:01PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 30 lines
Quoted 20 lines
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1263
Quoted 4 lines
kernel/kprobes.c +1
-0@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);Can you also set the
kcb->kprobe_state=KPROBE_HIT_ACTIVE?BTW, we may be better to introduce a common
kprobe_reject_section_start()andkprobe_reject_section_end()so that the user don't need to prepare dummy kprobes.
Quoted 8 lines
Quoted 3 lines
sure, will do
thank you both for review
ok, found out it's actually arch code.. would you guys be ok with something like below?
jirka
Hi Jiri,
In my origin test lockup happens on both x86_64 and ppc64le. So I would appreciate if you can also come up with a solution for both of the architectures.
Quoted 107 lines
---
arch/x86/kernel/kprobes/core.c +24
-13@@ -757,12 +757,33 @@ static struct kprobe kretprobe_kprobe = { .addr = (void *)kretprobe_trampoline, }; +void arch_kprobe_reject_section_start (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable (); + +/* +* Set a dummy kprobe for avoiding kretprobe recursion. +* Since kretprobe never run in kprobe handler, kprobe must not +* be running behind this point. +*/ +__this_cpu_write (current_kprobe, &kretprobe_kprobe); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +} + +void arch_kprobe_reject_section_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;@@ -772,16 +793,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); - -/* -* Set a dummy kprobe for avoiding kretprobe recursion. -* Since kretprobe never run in kprobe handler, kprobe must not -* be running at this point. -*/ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +arch_kprobe_reject_section_start ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);@@ -873,8 +885,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +arch_kprobe_reject_section_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);kernel/kprobes.c +12
-0@@ -1236,6 +1236,14 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +void __weak arch_kprobe_reject_section_start (void ) +{ +} + +void __weak arch_kprobe_reject_section_end (void ) +{ +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated @@ -1253,6 +1261,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +arch_kprobe_reject_section_start (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];@@ -1266,6 +1276,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +arch_kprobe_reject_section_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);
Signature · 6 lines
Ziqian SUN (Zamir) 9F Raycom office (NAY) Red Hat Software (Beijing) Co.,Ltd IRC: zsun (internal and freenode) Tel: +86 10 65627458 GPG : 1D86 6D4A 49CE 4BBD 72CF FCF5 D856 6E11 F2A0 525E
Quoted 44 lines · Ziqian SUN (Zamir)
Quoted 41 lines · Jiri Olsa
Quoted 30 lines · Jiri Olsa
On Thu, Apr 09, 2020 at 11:41:01PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 26 lines
Quoted 19 lines
--- kernel/kprobes.c | 6 ++++++ 1 file changed, 6 insertions(+)kernel/kprobes.c +4
-0@@ -1236,6 +1236,10 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +static struct kprobe kretprobe_dummy = { + .addr = (void *)kretprobe_trampoline, +}; +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated kernel/kprobes.c void kprobe_flush_task(struct task_struct *tk) · line 1263
Quoted 4 lines
kernel/kprobes.c +1
-0@@ -1256,12 +1260,14 @@ void kprobe_flush_task(struct task_struct *tk)INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash]; +__this_cpu_write (current_kprobe, &kretprobe_dummy);Can you also set the
kcb->kprobe_state=KPROBE_HIT_ACTIVE?Quoted 7 lines
Quoted 3 lines
BTW, we may be better to introduce a common
kprobe_reject_section_start()andkprobe_reject_section_end()so that the user don't need to prepare dummy kprobes.sure, will do
thank you both for review
ok, found out it's actually arch code.. would you guys be ok with something like below?
Quoted 8 lines
Quoted 2 lines
jirka
Hi Jiri,
In my origin test lockup happens on both
x86_64and ppc64le. So I would appreciate if you can also come up with a solution for both of the architectures.
aaaah right.. will update the fix
thanks,
jirka
On Fri, Apr 10, 2020 at 09:31:59AM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 26 lines
Quoted 13 lines
arch/x86/kernel/kprobes/core.c +5
-0@@ -757,12 +757,33 @@ static struct kprobe kretprobe_kprobe = { .addr = (void *)kretprobe_trampoline, }; +void arch_kprobe_reject_section_start (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable ();arch/x86/kernel/kprobes/core.c static struct kprobe kretprobe_kprobe = { · line 774
Quoted 10 lines
arch/x86/kernel/kprobes/core.c +10
-0+ +/* +* Set a dummy kprobe for avoiding kretprobe recursion. +* Since kretprobe never run in kprobe handler, kprobe must not +* be running behind this point. +*/ +__this_cpu_write (current_kprobe, &kretprobe_kprobe); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +}Yeah, the code seems good to me.
Quoted 4 lines
BTW, I rather like make it arch independent function so that other arch can use it. In this case, the dummy kprobe's addr should be somewhere obviously blacklisted (but it must be accessible.) I think
get_kprobe()will be a candidate.
right.. as Ziqian noted we see this on other ppc as well
Quoted 4 lines
And (sorry about changing my mind), the naming, I think
kprobe_busy_begin()andkprobe_busy_end()will be better because it doesn't reject registering kprobes, instead, just make it busy :)
ok, will change
thanks,
jirka
PATCH 1/1 kretprobe: Prevent triggering kretprobe from within kprobe_flush_task
Ziqian reported lockup when adding retprobe on _raw_spin_lock_irqsave. My test was also able to trigger lockdep output:
============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_hash_lock+0x52/0xa0
but task is already holding lock:
ffffffff9a491a18 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_trampoline+0x0/0x50
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
---- lock(&(kretprobe_table_locks[i].lock));
lock(&(kretprobe_table_locks[i].lock));
*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0: ffffffff9a491a18 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_trampoline+0x0/0x50
stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any _raw_spin_lock_irqsave probe in there.
The problem is in outside kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsave
where _raw_spin_lock_irqsave triggers the kretprobe and installs kretprobe_trampoline handler on _raw_spin_lock_irqsave return.
The kretprobe_trampoline handler is then executed with already locked kretprobe_table_locks, and first thing it does is to lock kretprobe_table_locks ;-) the whole lockup path like:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed
---> kretprobe_table_locks locked
kretprobe_trampoline
trampoline_handler
kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Using these helpers in kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.
--- arch/x86/kernel/kprobes/core.c | 16 +++------------- include/linux/kprobes.h | 4 ++++ kernel/kprobes.c | 24 ++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 13 deletions(-)
arch/x86/kernel/kprobes/core.c +3 -13
@@ -753,16 +753,11 @@ asm(NOKPROBE_SYMBOL (kretprobe_trampoline);STACK_FRAME_NON_STANDARD (kretprobe_trampoline); -static struct kprobe kretprobe_kprobe = { - .addr = (void *)kretprobe_trampoline, -}; -/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;
@@ -772,16 +767,12 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); -/* * Set a dummy kprobe for avoiding kretprobe recursion. * Since kretprobe never run in kprobe handler, kprobe must not * be running at this point. */ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +kprobe_busy_begin ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);
@@ -857,7 +848,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)__this_cpu_write (current_kprobe, &ri->rp->kp); ri->ret_addr = correct_ret_addr; ri->rp->handler (ri, regs); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); +__this_cpu_write (current_kprobe, &kprobe_busy); }recycle_rp_inst (ri, &empty_rp);
@@ -873,8 +864,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +kprobe_busy_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
include/linux/kprobes.h +4 -0
@@ -350,6 +350,10 @@ static inline struct kprobe_ctlblk *get_kprobe_ctlblk(void)return this_cpu_ptr (&kprobe_ctlblk); } +extern struct kprobe kprobe_busy; +void kprobe_busy_begin (void ); +void kprobe_busy_end (void ); +kprobe_opcode_t *kprobe_lookup_name (const char *name,unsigned int offset);int register_kprobe (struct kprobe *p);void unregister_kprobe (struct kprobe *p);
kernel/kprobes.c +24 -0
@@ -1236,6 +1236,26 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +struct kprobe kprobe_busy = { + .addr = (void *) get_kprobe, +}; + +void kprobe_busy_begin (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable (); +__this_cpu_write (current_kprobe, &kprobe_busy); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +} + +void kprobe_busy_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated
@@ -1253,6 +1273,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +kprobe_busy_begin (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];
@@ -1266,6 +1288,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +kprobe_busy_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);
Signature · 1 line
2.18.2
Quoted 85 lines · On Wed, 15 Apr 2020 11:05:07 +0200 Jiri Olsa <[email protected]> wrote:
Ziqian reported lockup when adding retprobe on
_raw_spin_lock_irqsave. My test was also able to trigger lockdep output:============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Quoted 6 lines
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.Reported-by "Ziqian SUN (Zamir)" <[email protected]>Signed-off-by Jiri Olsa <[email protected]>
Thanks Jiri and Ziqian!
Looks good to me.
BTW, this is a kind of bugfix. So should it add a Fixes tag?
Thank you,
Quoted 131 lines
--- arch/x86/kernel/kprobes/core.c | 16 +++------------- include/linux/kprobes.h | 4 ++++ kernel/kprobes.c | 24 ++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 13 deletions(-)arch/x86/kernel/kprobes/core.c +3
-13@@ -753,16 +753,11 @@ asm(NOKPROBE_SYMBOL (kretprobe_trampoline);STACK_FRAME_NON_STANDARD (kretprobe_trampoline); -static struct kprobe kretprobe_kprobe = { - .addr = (void *)kretprobe_trampoline, -}; -/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;@@ -772,16 +767,12 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); -/* * Set a dummy kprobe for avoiding kretprobe recursion. * Since kretprobe never run in kprobe handler, kprobe must not * be running at this point. */ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +kprobe_busy_begin ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);@@ -857,7 +848,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)__this_cpu_write (current_kprobe, &ri->rp->kp); ri->ret_addr = correct_ret_addr; ri->rp->handler (ri, regs); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); +__this_cpu_write (current_kprobe, &kprobe_busy); }recycle_rp_inst (ri, &empty_rp);@@ -873,8 +864,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +kprobe_busy_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);include/linux/kprobes.h +4
-0@@ -350,6 +350,10 @@ static inline struct kprobe_ctlblk *get_kprobe_ctlblk(void)return this_cpu_ptr (&kprobe_ctlblk); } +extern struct kprobe kprobe_busy; +void kprobe_busy_begin (void ); +void kprobe_busy_end (void ); +kprobe_opcode_t *kprobe_lookup_name (const char *name,unsigned int offset);int register_kprobe (struct kprobe *p);void unregister_kprobe (struct kprobe *p);kernel/kprobes.c +24
-0@@ -1236,6 +1236,26 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +struct kprobe kprobe_busy = { + .addr = (void *) get_kprobe, +}; + +void kprobe_busy_begin (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable (); +__this_cpu_write (current_kprobe, &kprobe_busy); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +} + +void kprobe_busy_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated @@ -1253,6 +1273,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +kprobe_busy_begin (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];@@ -1266,6 +1288,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +kprobe_busy_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);Signature · 1 line
2.18.2
Signature · 1 line
Masami Hiramatsu <[email protected]>
On Thu, Apr 16, 2020 at 10:55:06AM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 16 lines
Quoted 7 lines
trampoline_handler
kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Quoted 6 lines
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.Reported-by "Ziqian SUN (Zamir)" <[email protected]>Signed-off-by Jiri Olsa <[email protected]>Thanks Jiri and Ziqian!
Quoted 8 lines
Looks good to me.
Acked-by Masami Hiramatsu <[email protected]>BTW, this is a kind of bugfix. So should it add a Fixes tag?
Fixes ef53d9c5e4da ('kprobes: improve kretprobe scalability with hashed locking')
ah right, do you want me to repost with those?
thanks,
jirka
PATCH v2 1/1 kretprobe: Prevent triggering kretprobe from within kprobe_flush_task
In reply to Masami HiramatsuZiqian reported lockup when adding retprobe on _raw_spin_lock_irqsave. My test was also able to trigger lockdep output:
============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_hash_lock+0x52/0xa0
but task is already holding lock:
ffffffff9a491a18 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_trampoline+0x0/0x50
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
---- lock(&(kretprobe_table_locks[i].lock));
lock(&(kretprobe_table_locks[i].lock));
*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0: ffffffff9a491a18 (&(kretprobe_table_locks[i].lock)){-.-.}, at: kretprobe_trampoline+0x0/0x50
stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any _raw_spin_lock_irqsave probe in there.
The problem is in outside kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsave
where _raw_spin_lock_irqsave triggers the kretprobe and installs kretprobe_trampoline handler on _raw_spin_lock_irqsave return.
The kretprobe_trampoline handler is then executed with already locked kretprobe_table_locks, and first thing it does is to lock kretprobe_table_locks ;-) the whole lockup path like:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed
---> kretprobe_table_locks locked
kretprobe_trampoline
trampoline_handler
kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Using these helpers in kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.
--- arch/x86/kernel/kprobes/core.c | 16 +++------------- include/linux/kprobes.h | 4 ++++ kernel/kprobes.c | 24 ++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 13 deletions(-)
v2 changes: updated changelog with Fixes/Ack and Cc stable
arch/x86/kernel/kprobes/core.c +3 -13
@@ -753,16 +753,11 @@ asm(NOKPROBE_SYMBOL (kretprobe_trampoline);STACK_FRAME_NON_STANDARD (kretprobe_trampoline); -static struct kprobe kretprobe_kprobe = { - .addr = (void *)kretprobe_trampoline, -}; -/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;
@@ -772,16 +767,12 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); -/* * Set a dummy kprobe for avoiding kretprobe recursion. * Since kretprobe never run in kprobe handler, kprobe must not * be running at this point. */ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +kprobe_busy_begin ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);
@@ -857,7 +848,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)__this_cpu_write (current_kprobe, &ri->rp->kp); ri->ret_addr = correct_ret_addr; ri->rp->handler (ri, regs); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); +__this_cpu_write (current_kprobe, &kprobe_busy); }recycle_rp_inst (ri, &empty_rp);
@@ -873,8 +864,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +kprobe_busy_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);
include/linux/kprobes.h +4 -0
@@ -350,6 +350,10 @@ static inline struct kprobe_ctlblk *get_kprobe_ctlblk(void)return this_cpu_ptr (&kprobe_ctlblk); } +extern struct kprobe kprobe_busy; +void kprobe_busy_begin (void ); +void kprobe_busy_end (void ); +kprobe_opcode_t *kprobe_lookup_name (const char *name,unsigned int offset);int register_kprobe (struct kprobe *p);void unregister_kprobe (struct kprobe *p);
kernel/kprobes.c +24 -0
@@ -1236,6 +1236,26 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +struct kprobe kprobe_busy = { + .addr = (void *) get_kprobe, +}; + +void kprobe_busy_begin (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable (); +__this_cpu_write (current_kprobe, &kprobe_busy); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +} + +void kprobe_busy_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated
@@ -1253,6 +1273,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +kprobe_busy_begin (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];
@@ -1266,6 +1288,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +kprobe_busy_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);
Signature · 1 line
2.18.2
Masami Hiramatsu
maintainerHi Jiri,
Quoted 22 lines · On Thu, 16 Apr 2020 11:13:20 +0200 Jiri Olsa <[email protected]> wrote:
On Thu, Apr 16, 2020 at 10:55:06AM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 18 lines
Quoted 7 lines
trampoline_handler
kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Quoted 6 lines
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.Reported-by "Ziqian SUN (Zamir)" <[email protected]>Signed-off-by Jiri Olsa <[email protected]>Thanks Jiri and Ziqian!
Looks good to me.
Quoted 8 lines
Quoted 6 lines
Acked-by Masami Hiramatsu <[email protected]>BTW, this is a kind of bugfix. So should it add a Fixes tag?
Fixes ef53d9c5e4da ('kprobes: improve kretprobe scalability with hashed locking')ah right, do you want me to repost with those?
Yeah, if you don't mind.
Thank you,
Quoted 4 lines
thanks,
jirka
Signature · 1 line
Masami Hiramatsu <[email protected]>
Masami Hiramatsu
maintainerQuoted 81 lines · On Thu, 16 Apr 2020 16:31:04 +0200 Jiri Olsa <[email protected]> wrote:
Ziqian reported lockup when adding retprobe on
_raw_spin_lock_irqsave. My test was also able to trigger lockdep output:============================================
WARNING: possible recursive locking detected
5.6.0-rc6+ #6 Not tainted
--------------------------------------------
sched-messaging/2767 is trying to acquire lock:
ffffffff9a492798(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_hash_lock+0x52/0xa0but task is already holding lock:
ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50other info that might help us debug this:
Possible unsafe locking scenario:CPU0 ---- lock(&(kretprobe_table_locks[i].lock)); lock(&(kretprobe_table_locks[i].lock));*** DEADLOCK ***
May be due to missing lock nesting notation
1 lock held by sched-messaging/2767:
#0:ffffffff9a491a18(&(kretprobe_table_locks[i].lock)){-.-.}, at:kretprobe_trampoline+0x0/0x50stack backtrace: CPU: 3 PID: 2767 Comm: sched-messaging Not tainted 5.6.0-rc6+ #6 Call Trace: dump_stack+0x96/0xe0 __lock_acquire.cold.57+0x173/0x2b7 ? native_queued_spin_lock_slowpath+0x42b/0x9e0 ? lockdep_hardirqs_on+0x590/0x590 ? __lock_acquire+0xf63/0x4030 lock_acquire+0x15a/0x3d0 ? kretprobe_hash_lock+0x52/0xa0 _raw_spin_lock_irqsave+0x36/0x70 ? kretprobe_hash_lock+0x52/0xa0 kretprobe_hash_lock+0x52/0xa0 trampoline_handler+0xf8/0x940 ? kprobe_fault_handler+0x380/0x380 ? find_held_lock+0x3a/0x1c0 kretprobe_trampoline+0x25/0x50 ? lock_acquired+0x392/0xbc0 ? _raw_spin_lock_irqsave+0x50/0x70 ? __get_valid_kprobe+0x1f0/0x1f0 ? _raw_spin_unlock_irqrestore+0x3b/0x40 ? finish_task_switch+0x4b9/0x6d0 ? __switch_to_asm+0x34/0x70 ? __switch_to_asm+0x40/0x70The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlock
Quoted 8 lines
Adding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.
Thanks Jiri!
Ingo, could you pick this up?
Regards,
Quoted 138 lines
Fixes ef53d9c5e4da ('kprobes: improve kretprobe scalability with hashed locking')Reported-by "Ziqian SUN (Zamir)" <[email protected]>Acked-by Masami Hiramatsu <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- arch/x86/kernel/kprobes/core.c | 16 +++------------- include/linux/kprobes.h | 4 ++++ kernel/kprobes.c | 24 ++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 13 deletions(-)v2 changes: updated changelog with Fixes/Ack and Cc stable
arch/x86/kernel/kprobes/core.c +3
-13@@ -753,16 +753,11 @@ asm(NOKPROBE_SYMBOL (kretprobe_trampoline);STACK_FRAME_NON_STANDARD (kretprobe_trampoline); -static struct kprobe kretprobe_kprobe = { - .addr = (void *)kretprobe_trampoline, -}; -/* * Called from kretprobe_trampoline */ __used __visible void *trampoline_handler(struct pt_regs *regs) { -struct kprobe_ctlblk *kcb;struct kretprobe_instance *ri =NULL ;struct hlist_head *head, empty_rp;struct hlist_node *tmp;@@ -772,16 +767,12 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)void *frame_pointer;bool skipped = false; -preempt_disable (); -/* * Set a dummy kprobe for avoiding kretprobe recursion. * Since kretprobe never run in kprobe handler, kprobe must not * be running at this point. */ - kcb =get_kprobe_ctlblk (); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); - kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +kprobe_busy_begin ();INIT_HLIST_HEAD (&empty_rp);kretprobe_hash_lock (current, &head, &flags);@@ -857,7 +848,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)__this_cpu_write (current_kprobe, &ri->rp->kp); ri->ret_addr = correct_ret_addr; ri->rp->handler (ri, regs); -__this_cpu_write (current_kprobe, &kretprobe_kprobe); +__this_cpu_write (current_kprobe, &kprobe_busy); }recycle_rp_inst (ri, &empty_rp);@@ -873,8 +864,7 @@ __used __visible void *trampoline_handler(struct pt_regs *regs)kretprobe_hash_unlock (current, &flags); -__this_cpu_write (current_kprobe,NULL ); -preempt_enable (); +kprobe_busy_end ();hlist_for_each_entry_safe (ri, tmp, &empty_rp, hlist) {hlist_del (&ri->hlist);include/linux/kprobes.h +4
-0@@ -350,6 +350,10 @@ static inline struct kprobe_ctlblk *get_kprobe_ctlblk(void)return this_cpu_ptr (&kprobe_ctlblk); } +extern struct kprobe kprobe_busy; +void kprobe_busy_begin (void ); +void kprobe_busy_end (void ); +kprobe_opcode_t *kprobe_lookup_name (const char *name,unsigned int offset);int register_kprobe (struct kprobe *p);void unregister_kprobe (struct kprobe *p);kernel/kprobes.c +24
-0@@ -1236,6 +1236,26 @@ __releases(hlist_lock) }NOKPROBE_SYMBOL (kretprobe_table_unlock); +struct kprobe kprobe_busy = { + .addr = (void *) get_kprobe, +}; + +void kprobe_busy_begin (void ) +{ +struct kprobe_ctlblk *kcb; + +preempt_disable (); +__this_cpu_write (current_kprobe, &kprobe_busy); + kcb =get_kprobe_ctlblk (); + kcb->kprobe_status =KPROBE_HIT_ACTIVE ; +} + +void kprobe_busy_end (void ) +{ +__this_cpu_write (current_kprobe,NULL ); +preempt_enable (); +} +/* * This function is called from finish_task_switch when task tk becomes dead, * so that we can recycle any function-return probe instances associated @@ -1253,6 +1273,8 @@ void kprobe_flush_task(struct task_struct *tk)/* Early boot. kretprobe_table_locks not yet initialized. */ return ; +kprobe_busy_begin (); +INIT_HLIST_HEAD (&empty_rp); hash =hash_ptr (tk,KPROBE_HASH_BITS ); head = &kretprobe_inst_table[hash];@@ -1266,6 +1288,8 @@ void kprobe_flush_task(struct task_struct *tk)hlist_del (&ri->hlist);kfree (ri); } + +kprobe_busy_end (); }NOKPROBE_SYMBOL (kprobe_flush_task);Signature · 1 line
2.18.2
Signature · 1 line
Masami Hiramatsu <[email protected]>
On Fri, Apr 17, 2020 at 04:38:10PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 33 lines
Quoted 33 lines
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Quoted 8 lines
Quoted 4 lines
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.Thanks Jiri!
Ingo, could you pick this up?
Ingo, any chance you could take this one?
thanks,
jirka
Masami Hiramatsu
maintainerQuoted 42 lines · On Tue, 28 Apr 2020 23:36:27 +0200 Jiri Olsa <[email protected]> wrote:
On Fri, Apr 17, 2020 at 04:38:10PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 38 lines
Quoted 37 lines
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockAdding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.
Quoted 5 lines
Quoted 3 lines
Thanks Jiri!
Ingo, could you pick this up?
Ingo, any chance you could take this one?
Hi Ingo,
Should I make a pull request for all kprobes related patches to you?
Thank you,
Signature · 1 line
Masami Hiramatsu <[email protected]>
Quoted 47 lines · Masami Hiramatsu
Quoted 44 lines · On Tue, 28 Apr 2020 23:36:27 +0200 Jiri Olsa <[email protected]> wrote:
On Fri, Apr 17, 2020 at 04:38:10PM +0900, Masami Hiramatsu wrote:
SNIP
Quoted 40 lines
Quoted 29 lines
The code within the kretprobe handler checks for probe reentrancy, so we won't trigger any
_raw_spin_lock_irqsaveprobe in there.The problem is in outside
kprobe_flush_task, where we call:
kprobe_flush_task
kretprobe_table_lock
raw_spin_lock_irqsave
_raw_spin_lock_irqsavewhere
_raw_spin_lock_irqsavetriggers the kretprobe and installskretprobe_trampolinehandler on_raw_spin_lock_irqsavereturn.The
kretprobe_trampolinehandler is then executed with already lockedkretprobe_table_locks, and first thing it does is to lockkretprobe_table_locks;-) the whole lockup path like:
kprobe_flush_taskkretprobe_table_lock raw_spin_lock_irqsave _raw_spin_lock_irqsave ---> probe triggered, kretprobe_trampoline installed ---> kretprobe_table_locks locked kretprobe_trampoline trampoline_handler kretprobe_hash_lock(current, &head, &flags); <--- deadlockQuoted 8 lines
Adding kprobe_busy_begin/end helpers that mark code with fake probe installed to prevent triggering of another kprobe within this code.
Using these helpers in
kprobe_flush_task, so the probe recursion protection check is hit and the probe is never set to prevent above lockup.Thanks Jiri!
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Ingo, could you pick this up?
Ingo, any chance you could take this one?
Hi Ingo,
Should I make a pull request for all kprobes related patches to you?
looks like Ingo is offline, Thomas, could you please pull this one?
thanks,
jirka