Skip to contentSkip to conversation

Uprobes: Fix kernel oops with delayed_uprobe_remove()

PATCH Superseded Ravi Bangoria 4 participants 9 messages Nov 14, 2018 – Dec 5, 2018

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Uprobes: Fix kernel oops with delayed_uprobe_remove()

syzbot reported a kernel crash with delayed_uprobe_remove():
https://lkml.org/lkml/2018/11/1/1244

Backtrace mentioned in the link points to a race between process exit and uprobe_unregister(). Fix it by locking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Reported-by syzbot+cb1fb754b771caca0a88 at syzkaller.appspotmail.com
---
 kernel/events/uprobes.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/events/uprobes.c +2 -0
index 96fb51f3994f..e527c4753d4f 100644
@@ -572,7 +572,9 @@ static void put_uprobe(struct uprobe *uprobe)
 		 * gets called, we don't get a chance to remove uprobe from
 		 * delayed_uprobe_list from remove_breakpoint(). Do it here.
 		 */
+		mutex_lock(&delayed_uprobe_lock);
 		delayed_uprobe_remove(uprobe, NULL);
+		mutex_unlock(&delayed_uprobe_lock);
 		kfree(uprobe);
 	}
 }
Signature · 1 line
2.19.1
maintainer
#1 lore raw
Quoted 7 lines · Ravi Bangoria

syzbot reported a kernel crash with delayed_uprobe_remove():
https://lkml.org/lkml/2018/11/1/1244

Backtrace mentioned in the link points to a race between process exit and uprobe_unregister(). Fix it by locking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

The patch looks good to me, but could you update the changelog?

Please explain that the exiting task calls uprobe_clear_state() which can race with delayed_uprobe_remove(). IIUC this is the only problem solved by this patch, right?

Oleg.

#2 lore raw

Hi Oleg,

Quoted 9 lines · Oleg Nesterov
Quoted 7 lines · Ravi Bangoria

syzbot reported a kernel crash with delayed_uprobe_remove():
https://lkml.org/lkml/2018/11/1/1244

Backtrace mentioned in the link points to a race between process exit and uprobe_unregister(). Fix it by locking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Quoted 5 lines

The patch looks good to me, but could you update the changelog?

Please explain that the exiting task calls uprobe_clear_state() which can race with delayed_uprobe_remove(). IIUC this is the only problem solved by this patch, right?

Right. Is this better:

There could be a race between task exit and probe unregister:

exit_mm()
mmput()

  __mmput()                     uprobe_unregister()
  uprobe_clear_state()          put_uprobe()
  delayed_uprobe_remove()       delayed_uprobe_remove()

put_uprobe() is calling delayed_uprobe_remove() without taking delayed_uprobe_lock and thus the race sometimes results in a kernel crash. Fix this by taking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Detailed crash log can be found at:
https://lkml.org/lkml/2018/11/1/1244

maintainer
#3 lore raw
Quoted 9 lines · Ravi Bangoria

There could be a race between task exit and probe unregister:

exit_mm()
mmput()

  __mmput()                     uprobe_unregister()
  uprobe_clear_state()          put_uprobe()
  delayed_uprobe_remove()       delayed_uprobe_remove()
Quoted 7 lines

put_uprobe() is calling delayed_uprobe_remove() without taking delayed_uprobe_lock and thus the race sometimes results in a kernel crash. Fix this by taking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Detailed crash log can be found at:
https://lkml.org/lkml/2018/11/1/1244

Thanks, looks good,

Oleg.

#4 lore raw

Hi Steve,

Please pull this patch.

Thanks.

Quoted 22 lines · Oleg Nesterov
Quoted 9 lines · Ravi Bangoria

There could be a race between task exit and probe unregister:

exit_mm()
mmput()

  __mmput()                     uprobe_unregister()
  uprobe_clear_state()          put_uprobe()
  delayed_uprobe_remove()       delayed_uprobe_remove()
Quoted 7 lines

put_uprobe() is calling delayed_uprobe_remove() without taking delayed_uprobe_lock and thus the race sometimes results in a kernel crash. Fix this by taking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Detailed crash log can be found at:
https://lkml.org/lkml/2018/11/1/1244

Thanks, looks good,

Oleg.

#5 lore raw
Quoted 4 lines · On Mon, 3 Dec 2018 11:52:41 +0530 Ravi Bangoria <[email protected]>…

Hi Steve,

Please pull this patch.

Please send a v2 version of the patch with the updated change log. And should it have a Fixes and be tagged for stable?

-- Steve

Quoted 25 lines

Thanks.

Quoted 22 lines · Oleg Nesterov
Quoted 9 lines · Ravi Bangoria

There could be a race between task exit and probe unregister:

exit_mm()
mmput()

  __mmput()                     uprobe_unregister()
  uprobe_clear_state()          put_uprobe()
  delayed_uprobe_remove()       delayed_uprobe_remove()
Quoted 7 lines

put_uprobe() is calling delayed_uprobe_remove() without taking delayed_uprobe_lock and thus the race sometimes results in a kernel crash. Fix this by taking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Detailed crash log can be found at:
https://lkml.org/lkml/2018/11/1/1244

Thanks, looks good,

Oleg.

#6 lore raw

PATCH v2 1/1 Uprobes: Fix kernel oops with delayed_uprobe_remove()

There could be a race between task exit and probe unregister:

exit_mm()
mmput()

  __mmput()                     uprobe_unregister()
  uprobe_clear_state()          put_uprobe()
  delayed_uprobe_remove()       delayed_uprobe_remove()

put_uprobe() is calling delayed_uprobe_remove() without taking delayed_uprobe_lock and thus the race sometimes results in a kernel crash. Fix this by taking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Detailed crash log can be found at:
https://lkml.org/lkml/2018/11/1/1244

Reported-by [email protected]
Fixes 1cc33161a83d ("uprobes: Support SDT markers having reference count (semaphore)")
---
 kernel/events/uprobes.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/events/uprobes.c +2 -0
index 322e97bbb437..abbd8da9ac21 100644
@@ -572,7 +572,9 @@ static void put_uprobe(struct uprobe *uprobe)
 		 * gets called, we don't get a chance to remove uprobe from
 		 * delayed_uprobe_list from remove_breakpoint(). Do it here.
 		 */
+		mutex_lock(&delayed_uprobe_lock);
 		delayed_uprobe_remove(uprobe, NULL);
+		mutex_unlock(&delayed_uprobe_lock);
 		kfree(uprobe);
 	}
 }
Signature · 1 line
2.19.2
Reviewed-by on v2 1/1
#7 lore raw
Quoted 13 lines · Ravi Bangoria

There could be a race between task exit and probe unregister:

exit_mm()
mmput()

  __mmput()                     uprobe_unregister()
  uprobe_clear_state()          put_uprobe()
  delayed_uprobe_remove()       delayed_uprobe_remove()

put_uprobe() is calling delayed_uprobe_remove() without taking delayed_uprobe_lock and thus the race sometimes results in a kernel crash. Fix this by taking delayed_uprobe_lock before calling delayed_uprobe_remove() from put_uprobe().

Quoted 6 lines

Detailed crash log can be found at:
https://lkml.org/lkml/2018/11/1/1244

Reported-by [email protected]
Fixes 1cc33161a83d ("uprobes: Support SDT markers having reference count (semaphore)")

Looks good to me.

Quoted 21 lines
---
 kernel/events/uprobes.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/events/uprobes.c +2 -0
index 322e97bbb437..abbd8da9ac21 100644
@@ -572,7 +572,9 @@ static void put_uprobe(struct uprobe *uprobe)
 		 * gets called, we don't get a chance to remove uprobe from
 		 * delayed_uprobe_list from remove_breakpoint(). Do it here.
 		 */
+		mutex_lock(&delayed_uprobe_lock);
 		delayed_uprobe_remove(uprobe, NULL);
+		mutex_unlock(&delayed_uprobe_lock);
 		kfree(uprobe);
 	}
 }
Signature · 1 line
2.19.2
Signature · 2 lines
Thanks and Regards
Srikar Dronamraju
maintainer Acked-by on v2 1/1
#8 lore raw
In reply to Ravi Bangoria

kernel/events/uprobes.c static void put_uprobe(struct uprobe *uprobe) · line 578

Quoted 11 lines · Ravi Bangoria
kernel/events/uprobes.c +2 -0
@@ -572,7 +572,9 @@ static void put_uprobe(struct uprobe *uprobe)
 		 * gets called, we don't get a chance to remove uprobe from
 		 * delayed_uprobe_list from remove_breakpoint(). Do it here.
 		 */
+		mutex_lock(&delayed_uprobe_lock);
 		delayed_uprobe_remove(uprobe, NULL);
+		mutex_unlock(&delayed_uprobe_lock);
 		kfree(uprobe);