uprobes: register/unregister bugfixes

Hello.

Misc bugfixes, mostly register/unregister related.

Note: 4/7 and especially 6/7 are the really ugly (but hopefully temporary) hacks. We already discussed this, we should inspect the original insn in uprobe_register(). But this is not easy, and we need something simple to fix (some of) the bugs right now.

Even 7/7 is ugly. Imho UPROBE_SKIP_SSTEP should die as well but this is another story.

Oleg.

 arch/x86/kernel/uprobes.c |   16 +------
 include/linux/uprobes.h   |   10 ----
 kernel/events/uprobes.c   |  107 +++++++++++++++++++++++++++++++-------------
 3 files changed, 77 insertions(+), 56 deletions(-)

This is on top of "[PATCH 0/3] uprobes: mprotect fixes" series, which is still waiting for review...

Quoted 4 lines

This is on top of "[PATCH 0/3] uprobes: mprotect fixes" series, which is still waiting for review...

Did you mean [PATCH 0/4] uprobes: remove is_swbp_at_addr() from register/unregister?

I have already reviewed and acked "[PATCH 0/3] uprobes: mprotect fixes" series".

Got distracted by the register/unregister bug that we were discussing offline.

Multiple threads can manipulate uprobe->flags, this is obviously unsafe. For example mmap can set UPROBE_COPY_INSN while register tries to set UPROBE_RUN_HANDLER, the latter can also race with can_skip_sstep() which clears UPROBE_SKIP_SSTEP.

Change this code to use bitops.

Signed-off-by Oleg Nesterov <[email protected]>

---
 kernel/events/uprobes.c |   28 ++++++++++++++--------------
 1 files changed, 14 insertions(+), 14 deletions(-)

__skip_sstep() correctly detects the "nontrivial" nop insns, but since it doesn't update regs->ip we can not really skip "0x0f 0x1f | 0x0f 0x19 | 0x87 0xc0", the probed application is killed by SIGILL'ed handle_swbp().

Remove these additional checks. If we want to implement this correctly we need to know the full insn length to update ->ip.

rep* + nop is fine even without updating ->ip.

Signed-off-by Oleg Nesterov <[email protected]>

---

If alloc_uprobe() fails uprobe_register() should return ENOMEM, not 0.

Signed-off-by Oleg Nesterov <[email protected]>

---

delete_uprobe() must not be called if register_for_each_vma(false) fails to remove all breakpoints, __uprobe_unregister() is correct. The problem is that register_for_each_vma(false) always returns 0 and thus this logic does not work.

1. Change verify_opcode() to return 0 rather than -EINVAL when unregister detects the !is_swbp insn, we can treat this case as success and currently unregister paths ignore the error code anyway.

2. Change remove_breakpoint() to propagate the error code from write_opcode().

3. Change register_for_each_vma(is_register => false) to remove as much breakpoints as possible but return non-zero if remove_breakpoint() fails at least once.

Signed-off-by Oleg Nesterov <[email protected]>

Strictly speaking this race was added by me in 56bb4cf6. However I think that this bug is just another indication that we should move copy_insn/uprobe_analyze_insn code from install_breakpoint() to uprobe_register(), there are a lot of other reasons for that. Until then, add a hack to close the race.

A task can hit uprobe U1, but before it calls find_uprobe() this uprobe can be unregistered *AND* another uprobe U2 can be added to uprobes_tree at the same inode/offset. In this case handle_swbp() will use the not-fully-initialized U2, in particular its arch.insn for xol.

Add the additional !UPROBE_COPY_INSN check into find_active_uprobe, if this flag is not set we simply restart as if the new uprobe was not inserted yet. This is not very nice, we need barriers, but we will remove this hack when we change uprobe_register().

Note: with or without this patch install_breakpoint() can race with itself, yet another reson to kill UPROBE_COPY_INSN altogether. And even the usage of uprobe->flags is not safe. See the next patches.

Signed-off-by Oleg Nesterov <[email protected]>

---
 kernel/events/uprobes.c |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

kernel/events/uprobes.c +4 -0

index 6058231..a81080f 100644
@@ -596,6 +596,7 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
 		BUG_ON((uprobe->offset & ~PAGE_MASK) +
 				UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
 
+		smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
 		uprobe->flags |= UPROBE_COPY_INSN;
 	}
 

@@ -1391,6 +1392,16 @@ static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp)
 	if (!uprobe && test_and_clear_bit(MMF_RECALC_UPROBES, &mm->flags))
 		mmf_recalc_uprobes(mm);
 	up_read(&mm->mmap_sem);
+	/*
+	 * TODO: move copy_insn/etc into _register and remove this hack.
+	 * After we hit the bp, _unregister + _register can install the

Preparation. Extract the copy_insn/arch_uprobe_analyze_insn code from install_breakpoint() into the new helper, uprobe_copy_insn().

And move uprobe->flags defines from uprobes.h to uprobes.c, nobody else can use them anyway.

Signed-off-by Oleg Nesterov <[email protected]>

---
 include/linux/uprobes.h |   10 --------
 kernel/events/uprobes.c |   60 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 41 insertions(+), 29 deletions(-)

include/linux/uprobes.h +0 -10

index 18d839d..2459457 100644
@@ -35,16 +35,6 @@ struct inode;
 # include <asm/uprobes.h>
 #endif
 
-/* flags that denote/change uprobes behaviour */
-
-/* Have a copy of original instruction */
-#define UPROBE_COPY_INSN	0x1
-
-/* Dont run handlers when first register/ last unregister in progress*/
-#define UPROBE_RUN_HANDLER	0x2
-/* Can skip singlestep */
-#define UPROBE_SKIP_SSTEP	0x4
-
 struct uprobe_consumer {
 	int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
 	/*

kernel/events/uprobes.c +28 -0

index a81080f..5c0c1b0 100644
@@ -78,6 +78,13 @@ static struct mutex uprobes_mmap_mutex[UPROBES_HASH_SZ];
  */
 static atomic_t uprobe_events = ATOMIC_INIT(0);
 
+/* Have a copy of original instruction */
+#define UPROBE_COPY_INSN	0x1
+/* Dont run handlers when first register/ last unregister in progress*/
+#define UPROBE_RUN_HANDLER	0x2
+/* Can skip singlestep */
+#define UPROBE_SKIP_SSTEP	0x4
+
 struct uprobe {
 	struct rb_node		rb_node;	/* node in the rb tree */
 	atomic_t		ref;

@@ -563,6 +570,37 @@ static int copy_insn(struct uprobe *uprobe, struct file *filp)
 	return __copy_insn(mapping, filp, uprobe->arch.insn, bytes, uprobe->offset);
 }
 
+static int uprobe_copy_insn(struct uprobe *uprobe, struct file *file,
+				struct mm_struct *mm, unsigned long vaddr)
+{
+	int ret = 0;
+
+	if (uprobe->flags & UPROBE_COPY_INSN)
+		return ret;
+
+	ret = copy_insn(uprobe, file);
+	if (ret)
+		goto out;
+
+	ret = -ENOTSUPP;
+	if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
+		goto out;
+
+	ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
+	if (ret)
+		goto out;
+
+	/* write_opcode() assumes we don't cross page boundary */

install_breakpoint() is called under mm->mmap_sem, this protects set_swbp() but not uprobe_copy_insn(). Two or more different tasks can call install_breakpoint()->uprobe_copy_insn() at the same time, this leads to numerous problems if UPROBE_COPY_INSN is not set.

Just for example, the second copy_insn() can corrupt the already analyzed/fixuped uprobe->arch.insn and race with handle_swbp().

This patch simply adds uprobe->copy_mutex to serialize this code. We could probably reuse ->consumer_rwsem, but this would mean that consumer->handler() can not use mm->mmap_sem, not good.

Note: this is another temporary ugly hack until we move this logic into uprobe_register().

Signed-off-by Oleg Nesterov <[email protected]>

Multiple threads can manipulate uprobe->flags, this is obviously unsafe. For example mmap can set UPROBE_COPY_INSN while register tries to set UPROBE_RUN_HANDLER, the latter can also race with can_skip_sstep() which clears UPROBE_SKIP_SSTEP.

Change this code to use bitops.

Signed-off-by Oleg Nesterov <[email protected]>

Quoted 27 lines · Oleg Nesterov

(quoted diff) +27 -0

+static int uprobe_copy_insn(struct uprobe *uprobe, struct file *file,
+				struct mm_struct *mm, unsigned long vaddr)
+{
+	int ret = 0;
+
+	if (uprobe->flags & UPROBE_COPY_INSN)
+		return ret;
+
+	ret = copy_insn(uprobe, file);
+	if (ret)
+		goto out;
+
+	ret = -ENOTSUPP;
+	if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
+		goto out;
+
+	ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
+	if (ret)
+		goto out;
+
+	/* write_opcode() assumes we don't cross page boundary */
+	BUG_ON((uprobe->offset & ~PAGE_MASK) +
+			UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
+
+	smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
+	uprobe->flags |= UPROBE_COPY_INSN;
+	ret = 0;

Review

Quoted 4 lines

+ out:

(quoted diff) +3 -0

+	return ret;
+}
+

2 nits:
why do we need to reset ret before out label? I think its redudant. arch_uprobe_analyze_insn() should have set it to 0 already. No?

blank line above out:

Currently only extern functions start with uprobe_ but we already have copy_insn, and __copy_insn, So can think of any names for uprobe_copy_insn.

Not sure test_and_copy_insn() is a good alternative.

Quoted 15 lines · Oleg Nesterov

kernel/events/uprobes.c +10 -0

@@ -1391,6 +1392,16 @@ static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp)
 	if (!uprobe && test_and_clear_bit(MMF_RECALC_UPROBES, &mm->flags))
 		mmf_recalc_uprobes(mm);
 	up_read(&mm->mmap_sem);
+	/*
+	 * TODO: move copy_insn/etc into _register and remove this hack.
+	 * After we hit the bp, _unregister + _register can install the
+	 * new and not-yet-analyzed uprobe at the same address, restart.
+	 */
+	smp_rmb(); /* pairs with wmb() in install_breakpoint() */
+	if (uprobe && unlikely(!(uprobe->flags & UPROBE_COPY_INSN))) {
+		uprobe = NULL;
+		*is_swbp = 0;
+	}
 

kernel/events/uprobes.c static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp) · line 1407

Quoted 2 lines

kernel/events/uprobes.c +0 -0

 	return uprobe;
 }

Should we be adding this check handle_swbp() around can_skip_step()?

The earliest we access arch.insn is in can_skip_step. So we give some more time for the instruction to be copied.

Also it will probably be a little cleaner, (Not having to drop a uprobe reference, not having to reset is_swbp.)

for the future changes... (say, we can remove bp if consumers do not want to trace this task). Not sure it makes sense to change it right now.

So. Should I leave this patch as is? Or do you want me to move this check into handler_chain() and make it return "bool restart"?

Quoted 1 line

uprobe_copy_insn.

Yes ;) plus it is not only "copy", it does _analyze

Quoted 1 line

Not sure test_and_copy_insn() is a good alternative.

perhaps prepare_uprobe()... But I agree with any naming, just tell me what you prefer and I'll update the patch again.

==============================================================================
[PATCH 5/7] uprobes: Introduce uprobe_copy_insn()

Preparation. Extract the copy_insn/arch_uprobe_analyze_insn code from install_breakpoint() into the new helper, uprobe_copy_insn().

And move uprobe->flags defines from uprobes.h to uprobes.c, nobody else can use them anyway.

Signed-off-by Oleg Nesterov <[email protected]>

---
 include/linux/uprobes.h |   10 --------
 kernel/events/uprobes.c |   60 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 41 insertions(+), 29 deletions(-)

include/linux/uprobes.h +0 -10

index 18d839d..2459457 100644
@@ -35,16 +35,6 @@ struct inode;
 # include <asm/uprobes.h>
 #endif
 
-/* flags that denote/change uprobes behaviour */
-
-/* Have a copy of original instruction */
-#define UPROBE_COPY_INSN	0x1
-
-/* Dont run handlers when first register/ last unregister in progress*/
-#define UPROBE_RUN_HANDLER	0x2
-/* Can skip singlestep */
-#define UPROBE_SKIP_SSTEP	0x4
-
 struct uprobe_consumer {
 	int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
 	/*

kernel/events/uprobes.c +41 -19

index dd44311..98b60bc 100644
@@ -78,6 +78,13 @@ static struct mutex uprobes_mmap_mutex[UPROBES_HASH_SZ];
  */
 static atomic_t uprobe_events = ATOMIC_INIT(0);
 
+/* Have a copy of original instruction */
+#define UPROBE_COPY_INSN	0x1
+/* Dont run handlers when first register/ last unregister in progress*/
+#define UPROBE_RUN_HANDLER	0x2
+/* Can skip singlestep */
+#define UPROBE_SKIP_SSTEP	0x4
+
 struct uprobe {
 	struct rb_node		rb_node;	/* node in the rb tree */
 	atomic_t		ref;

@@ -563,6 +570,37 @@ static int copy_insn(struct uprobe *uprobe, struct file *filp)
 	return __copy_insn(mapping, filp, uprobe->arch.insn, bytes, uprobe->offset);
 }
 
+static int uprobe_copy_insn(struct uprobe *uprobe, struct file *file,
+				struct mm_struct *mm, unsigned long vaddr)
+{
+	int ret = 0;
+
+	if (uprobe->flags & UPROBE_COPY_INSN)
+		return ret;
+
+	ret = copy_insn(uprobe, file);
+	if (ret)
+		goto out;
+
+	ret = -ENOTSUPP;
+	if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
+		goto out;
+
+	ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
+	if (ret)
+		goto out;
+
+	/* write_opcode() assumes we don't cross page boundary */
+	BUG_ON((uprobe->offset & ~PAGE_MASK) +
+			UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
+
+	smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
+	uprobe->flags |= UPROBE_COPY_INSN;
+
+ out:
+	return ret;
+}
+
 static int
 install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
 			struct vm_area_struct *vma, unsigned long vaddr)

@@ -580,25 +618,9 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
 	if (!uprobe->consumers)
 		return 0;
 
-	if (!(uprobe->flags & UPROBE_COPY_INSN)) {
-		ret = copy_insn(uprobe, vma->vm_file);
-		if (ret)
-			return ret;
-
-		if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
-			return -ENOTSUPP;
-
-		ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
-		if (ret)
-			return ret;
-
-		/* write_opcode() assumes we don't cross page boundary */
-		BUG_ON((uprobe->offset & ~PAGE_MASK) +
-				UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
-
-		smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
-		uprobe->flags |= UPROBE_COPY_INSN;
-	}
+	ret = uprobe_copy_insn(uprobe, vma->vm_file, mm, vaddr);
+	if (ret)
+		return ret;
 
 	/*
 	 * set MMF_HAS_UPROBES in advance for uprobe_pre_sstep_notifier(),

Signature · 1 line

1.5.5.1

Quoted 6 lines

for the future changes... (say, we can remove bp if consumers do not want to trace this task). Not sure it makes sense to change it right now.

Quoted 2 lines

So. Should I leave this patch as is? Or do you want me to move this check into handler_chain() and make it return "bool restart"?

Lets keep it as is for now.

Acked-by Srikar Dronamraju <[email protected]>

Yeah prepare_uprobe() looks good for me.

Acked-by Srikar Dronamraju <[email protected]>

Quoted 6 lines · Srikar Dronamraju

Quoted 6 lines

for the future changes... (say, we can remove bp if consumers do not want to trace this task). Not sure it makes sense to change it right now.

Quoted 6 lines

Quoted 2 lines

So. Should I leave this patch as is? Or do you want me to move this check into handler_chain() and make it return "bool restart"?

Lets keep it as is for now.

Acked-by Srikar Dronamraju <[email protected]>

Thanks...

But I am starting to think that I misunderstood your comment, you did not suggest to add this check into skip_sstep() as I wrongly thought.

And yes, I agree it would be more clean to move it out from find_active_uprobe() and avoid put_uprobe && clear_swbp....

So how about v2 below?

------------------------------------------------------------------------------
[PATCH 4/7] uprobes: Fix handle_swbp() vs unregister() + register() race

Strictly speaking this race was added by me in 56bb4cf6. However I think that this bug is just another indication that we should move copy_insn/uprobe_analyze_insn code from install_breakpoint() to uprobe_register(), there are a lot of other reasons for that. Until then, add a hack to close the race.

A task can hit uprobe U1, but before it calls find_uprobe() this uprobe can be unregistered *AND* another uprobe U2 can be added to uprobes_tree at the same inode/offset. In this case handle_swbp() will use the not-fully-initialized U2, in particular its arch.insn for xol.

Add the additional !UPROBE_COPY_INSN check into handle_swbp(), if this flag is not set we simply restart as if the new uprobe was not inserted yet. This is not very nice, we need barriers, but we will remove this hack when we change uprobe_register().

Note: with or without this patch install_breakpoint() can race with itself, yet another reson to kill UPROBE_COPY_INSN altogether. And even the usage of uprobe->flags is not safe. See the next patches.

Signed-off-by Oleg Nesterov <[email protected]>

Acked-by Srikar Dronamraju <[email protected]>

---
 kernel/events/uprobes.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

kernel/events/uprobes.c +9 -0

index cfa22c4..dbbca3a 100644
@@ -596,6 +596,7 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
 		BUG_ON((uprobe->offset & ~PAGE_MASK) +
 				UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
 
+		smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
 		uprobe->flags |= UPROBE_COPY_INSN;
 	}
 

@@ -1436,6 +1437,14 @@ static void handle_swbp(struct pt_regs *regs)
 		}
 		return;
 	}
+	/*
+	 * TODO: move copy_insn/etc into _register and remove this hack.
+	 * After we hit the bp, _unregister + _register can install the
+	 * new and not-yet-analyzed uprobe at the same address, restart.
+	 */
+	smp_rmb(); /* pairs with wmb() in install_breakpoint() */
+	if (unlikely(!(uprobe->flags & UPROBE_COPY_INSN)))
+		goto restart;
 
 	utask = current->utask;
 	if (!utask) {

Signature · 1 line

1.5.5.1

Quoted 4 lines · Srikar Dronamraju

Yeah prepare_uprobe() looks good for me.

Acked-by Srikar Dronamraju <[email protected]>

OK, renamed.

The next patches updated accordinly, I hope I can keep your acks.

------------------------------------------------------------------------------
[PATCH] uprobes: Introduce prepare_uprobe()

Preparation. Extract the copy_insn/arch_uprobe_analyze_insn code from install_breakpoint() into the new helper, prepare_uprobe().

And move uprobe->flags defines from uprobes.h to uprobes.c, nobody else can use them anyway.

Signed-off-by Oleg Nesterov <[email protected]>

Acked-by Srikar Dronamraju <[email protected]>

---
 include/linux/uprobes.h |   10 --------
 kernel/events/uprobes.c |   60 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 41 insertions(+), 29 deletions(-)

include/linux/uprobes.h +0 -10

index 18d839d..2459457 100644
@@ -35,16 +35,6 @@ struct inode;
 # include <asm/uprobes.h>
 #endif
 
-/* flags that denote/change uprobes behaviour */
-
-/* Have a copy of original instruction */
-#define UPROBE_COPY_INSN	0x1
-
-/* Dont run handlers when first register/ last unregister in progress*/
-#define UPROBE_RUN_HANDLER	0x2
-/* Can skip singlestep */
-#define UPROBE_SKIP_SSTEP	0x4
-
 struct uprobe_consumer {
 	int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
 	/*

kernel/events/uprobes.c +41 -19

index dbbca3a..d6fa332 100644
@@ -78,6 +78,13 @@ static struct mutex uprobes_mmap_mutex[UPROBES_HASH_SZ];
  */
 static atomic_t uprobe_events = ATOMIC_INIT(0);
 
+/* Have a copy of original instruction */
+#define UPROBE_COPY_INSN	0x1
+/* Dont run handlers when first register/ last unregister in progress*/
+#define UPROBE_RUN_HANDLER	0x2
+/* Can skip singlestep */
+#define UPROBE_SKIP_SSTEP	0x4
+
 struct uprobe {
 	struct rb_node		rb_node;	/* node in the rb tree */
 	atomic_t		ref;

@@ -563,6 +570,37 @@ static int copy_insn(struct uprobe *uprobe, struct file *filp)
 	return __copy_insn(mapping, filp, uprobe->arch.insn, bytes, uprobe->offset);
 }
 
+static int prepare_uprobe(struct uprobe *uprobe, struct file *file,
+				struct mm_struct *mm, unsigned long vaddr)
+{
+	int ret = 0;
+
+	if (uprobe->flags & UPROBE_COPY_INSN)
+		return ret;
+
+	ret = copy_insn(uprobe, file);
+	if (ret)
+		goto out;
+
+	ret = -ENOTSUPP;
+	if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
+		goto out;
+
+	ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
+	if (ret)
+		goto out;
+
+	/* write_opcode() assumes we don't cross page boundary */
+	BUG_ON((uprobe->offset & ~PAGE_MASK) +
+			UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
+
+	smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
+	uprobe->flags |= UPROBE_COPY_INSN;
+
+ out:
+	return ret;
+}
+
 static int
 install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
 			struct vm_area_struct *vma, unsigned long vaddr)

@@ -580,25 +618,9 @@ install_breakpoint(struct uprobe *uprobe, struct mm_struct *mm,
 	if (!uprobe->consumers)
 		return 0;
 
-	if (!(uprobe->flags & UPROBE_COPY_INSN)) {
-		ret = copy_insn(uprobe, vma->vm_file);
-		if (ret)
-			return ret;
-
-		if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
-			return -ENOTSUPP;
-
-		ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
-		if (ret)
-			return ret;
-
-		/* write_opcode() assumes we don't cross page boundary */
-		BUG_ON((uprobe->offset & ~PAGE_MASK) +
-				UPROBE_SWBP_INSN_SIZE > PAGE_SIZE);
-
-		smp_wmb(); /* pairs with rmb() in find_active_uprobe() */
-		uprobe->flags |= UPROBE_COPY_INSN;
-	}
+	ret = prepare_uprobe(uprobe, vma->vm_file, mm, vaddr);
+	if (ret)
+		return ret;
 
 	/*
 	 * set MMF_HAS_UPROBES in advance for uprobe_pre_sstep_notifier(),

Signature · 1 line

1.5.5.1