Skip to contentSkip to conversation

uprobes: return probe implementation

RFC PATCH Superseded Anton Arapov 2 participants 11 messages Jan 9, 2013 – Jan 10, 2013

Subsystems (3)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover uprobes: return probe implementation

There are RFC uretprobes implementation. I'd be grateful for review.

RFCv1: https://lkml.org/lkml/2012/12/21/133

I've intentionally removed the retprobe bypass logic, it requires a bit more work.

not fixed since last prior RFC review:
unify xol_get_trampoline_slot() and xol_take_insn_slot() protect uprobe in prepare_uretprobe()

v2 changes:
introduced rp_handler(), get rid of return_consumers get rid of uretprobe_[un]register() introduced arch_uretprobe_get_sp()
removed uprobe_task->doomed, kill task immediately fix arch_uretprobe_hijack_return_addr()'s returns address the v1 minor issues

thanks,

Anton Arapov (5):
uretprobes/x86: hijack return address
uretprobes: trampoline implementation
uretprobes: return probe entry, prepare uretprobe
uretprobes: invoke return probe handlers

 arch/x86/include/asm/uprobes.h |   6 ++
 arch/x86/kernel/uprobes.c      |  48 +++++++++++
 include/linux/uprobes.h        |   9 ++
 kernel/events/uprobes.c        | 181 ++++++++++++++++++++++++++++++++++++++++-
 4 files changed, 241 insertions(+), 3 deletions(-)
Signature · 1 line
1.8.0.2
#1 lore raw

RFC PATCH v2 1/4 uretprobes/x86: hijack return address

Function to hijack the return address, replace it with a "trampoline" and function to predict the stack pointer address value at return.

v2:
remove ->doomed flag, kill task immediately

---
 arch/x86/include/asm/uprobes.h |  1 +
 arch/x86/kernel/uprobes.c      | 29 +++++++++++++++++++++++++++++
 2 files changed, 30 insertions(+)
arch/x86/include/asm/uprobes.h +1 -0
index 8ff8be7..c353555 100644
@@ -55,4 +55,5 @@ extern int  arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);
 extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);
 extern int  arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data);
 extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
+extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long rp_trampoline_vaddr, struct pt_regs *regs);
 #endif	/* _ASM_UPROBES_H */
arch/x86/kernel/uprobes.c +29 -0
index 0ba4cfb..85e2153 100644
@@ -697,3 +697,32 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
 		send_sig(SIGTRAP, current, 0);
 	return ret;
 }
+
+extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long
+		rp_trampoline_vaddr, struct pt_regs *regs)
+{
+	int rasize, ncopied;
+	unsigned long orig_ret_vaddr = 0; /* clear high bits for 32-bit apps */
+
+	rasize = is_ia32_task() ? 4 : 8;
+	ncopied = copy_from_user(&orig_ret_vaddr, (void __user *)regs->sp, rasize);
+	if (unlikely(ncopied))
+		return 0;
+
+	/* check whether address has been already hijacked */
+	if (orig_ret_vaddr == rp_trampoline_vaddr)
+		return orig_ret_vaddr;
+
+	ncopied = copy_to_user((void __user *)regs->sp, &rp_trampoline_vaddr, rasize);
+	if (unlikely(ncopied)) {
+		if (ncopied != rasize) {
+			printk(KERN_ERR "uretprobe: return address clobbered: "
+					"pid=%d, %%sp=%#lx, %%ip=%#lx\n",
+					current->pid, regs->sp, regs->ip);
+			/* kill task immediately */
+			send_sig(SIGSEGV, current, 0);
+		}
+	}
+
+	return orig_ret_vaddr;
+}
Signature · 1 line
1.8.0.2
#2 lore raw

RFC PATCH v2 2/4 uretprobes: trampoline implementation

In reply to Anton Arapov

The first time a uprobe with return consumer is hit for a process, a trampoline slot is obtained in the xol_area and initialized with a breakpoint instruction. This slot is subsequently used by all uretprobes.

todo:
unify with xol_take_insn_slot()

---
 include/linux/uprobes.h |  1 +
 kernel/events/uprobes.c | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
include/linux/uprobes.h +1 -0
index 95d0002..bddfad6 100644
@@ -91,6 +91,7 @@ struct xol_area {
 	 * the vma go away, and we must handle that reasonably gracefully.
 	 */
 	unsigned long 		vaddr;		/* Page(s) of instruction slots */
+	unsigned long		rp_trampoline_vaddr; /* address of trampolines */
 };
 
 struct uprobes_state {
kernel/events/uprobes.c +33 -0
index 473db3f..0ad2ac3 100644
@@ -1208,6 +1208,39 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)
 	return xol_vaddr;
 }
 
+/*
+ * xol_get_trampoline_slot - A trampoline slot is obtained the first
+ * time a uprobe with return consumer is hit for a process. Use one
+ * trampoline slot for all probes pertaining to a process, i.e.
+ * one per xol_area.
+ */
+static unsigned long xol_get_trampoline_slot(void)
+{
+	struct xol_area *area;
+	unsigned long offset;
+	uprobe_opcode_t bp_insn = UPROBE_SWBP_INSN;
+	void *vaddr;
+
+	area = get_xol_area();
+	if (!area)
+		return 0;
+
+	if (!area->rp_trampoline_vaddr)
+		area->rp_trampoline_vaddr = xol_take_insn_slot(area);
+
+	if (unlikely(!area->rp_trampoline_vaddr))
+		return 0;
+
+	offset = area->rp_trampoline_vaddr & ~PAGE_MASK;
+	vaddr = kmap_atomic(area->page);
+	memcpy(vaddr + offset, &bp_insn, UPROBE_SWBP_INSN_SIZE);
+	kunmap_atomic(vaddr);
+
+	flush_dcache_page(area->page);
+
+	return area->rp_trampoline_vaddr;
+}
+
 /*
  * xol_free_insn_slot - If slot was earlier allocated by
  * @xol_get_insn_slot(), make the slot available for
Signature · 1 line
1.8.0.2
maintainer
#3 lore raw

Re: [RFC PATCH v2 2/4] uretprobes: trampoline implementation

Quoted 5 lines · Anton Arapov

The first time a uprobe with return consumer is hit for a process, a trampoline slot is obtained in the xol_area and initialized with a breakpoint instruction. This slot is subsequently used by all uretprobes.

See my reply to 0/4, I think we do not really need this.

Quoted 68 lines

todo:
unify with xol_take_insn_slot()

---
 include/linux/uprobes.h |  1 +
 kernel/events/uprobes.c | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
include/linux/uprobes.h +1 -0
index 95d0002..bddfad6 100644
@@ -91,6 +91,7 @@ struct xol_area {
 	 * the vma go away, and we must handle that reasonably gracefully.
 	 */
 	unsigned long 		vaddr;		/* Page(s) of instruction slots */
+	unsigned long		rp_trampoline_vaddr; /* address of trampolines */
 };
 
 struct uprobes_state {
kernel/events/uprobes.c +33 -0
index 473db3f..0ad2ac3 100644
@@ -1208,6 +1208,39 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)
 	return xol_vaddr;
 }
 
+/*
+ * xol_get_trampoline_slot - A trampoline slot is obtained the first
+ * time a uprobe with return consumer is hit for a process. Use one
+ * trampoline slot for all probes pertaining to a process, i.e.
+ * one per xol_area.
+ */
+static unsigned long xol_get_trampoline_slot(void)
+{
+	struct xol_area *area;
+	unsigned long offset;
+	uprobe_opcode_t bp_insn = UPROBE_SWBP_INSN;
+	void *vaddr;
+
+	area = get_xol_area();
+	if (!area)
+		return 0;
+
+	if (!area->rp_trampoline_vaddr)
+		area->rp_trampoline_vaddr = xol_take_insn_slot(area);
+
+	if (unlikely(!area->rp_trampoline_vaddr))
+		return 0;
+
+	offset = area->rp_trampoline_vaddr & ~PAGE_MASK;
+	vaddr = kmap_atomic(area->page);
+	memcpy(vaddr + offset, &bp_insn, UPROBE_SWBP_INSN_SIZE);
+	kunmap_atomic(vaddr);
+
+	flush_dcache_page(area->page);
+
+	return area->rp_trampoline_vaddr;
+}
+
 /*
  * xol_free_insn_slot - If slot was earlier allocated by
  * @xol_get_insn_slot(), make the slot available for
Signature · 1 line
1.8.0.2
#4 lore raw

RFC PATCH v2 3/4 uretprobes: return probe entry, prepare uretprobe

In reply to Anton Arapov

When a uprobe with return consumer is hit, prepare_uretprobe function is invoked. It creates return_instance, hijacks return address and replaces it with the trampoline.

v2:
get rid of ->return_consumers

todo:
protect uprobe

---
 include/linux/uprobes.h |  4 ++++
 kernel/events/uprobes.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 50 insertions(+), 2 deletions(-)
include/linux/uprobes.h +4 -0
index bddfad6..cdc4d53 100644
@@ -68,6 +68,10 @@ struct uprobe_task {
 	enum uprobe_task_state		state;
 	struct arch_uprobe_task		autask;
 
+	/*
+	 * list for tracking uprobes with return consumers
+	 */
+	struct hlist_head		return_uprobes;
 	struct uprobe			*active_uprobe;
 
 	unsigned long			xol_vaddr;
kernel/events/uprobes.c +46 -2

75 lines not on this page · raw message

Signature · 1 line
1.8.0.2
maintainer
#5 lore raw

Re: [RFC PATCH v2 3/4] uretprobes: return probe entry, prepare uretprobe

Quoted 3 lines · Anton Arapov

todo:
protect uprobe

Yep ;)

kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1338

Quoted 6 lines

static struct uprobe_task *get_utask(void)
{

kernel/events/uprobes.c +2 -1
-	if (!current->utask)
+	if (!current->utask) {
 		current->utask = kzalloc(sizeof(struct uprobe_task), GFP_KERNEL);
+		INIT_HLIST_HEAD(&current->utask->return_uprobes);

What if kzalloc() fails?

kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1354

Quoted 12 lines
kernel/events/uprobes.c +12 -0
+static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)
+{
+	struct return_uprobe_i *ri;
+	struct uprobe_task *utask;
+	struct xol_area *area;
+	unsigned long rp_trampoline_vaddr = 0;
+
+	area = get_xol_area();
+	if (area)
+		rp_trampoline_vaddr = area->rp_trampoline_vaddr;
+	if (!rp_trampoline_vaddr) {
+		rp_trampoline_vaddr = xol_get_trampoline_slot();

This is obviously racy. But again, so far I think we can simply remove this.

Quoted 4 lines

static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) {

kernel/events/uprobes.c +1 -0
+	int rc = 0;
 	struct uprobe_consumer *uc;

kernel/events/uprobes.c static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp) · line 1537

Quoted 10 lines
kernel/events/uprobes.c +5 -1
 	int remove = UPROBE_HANDLER_REMOVE;
 
 	down_read(&uprobe->register_rwsem);
 	for (uc = uprobe->consumers; uc; uc = uc->next) {
-		int rc = uc->handler(uc, regs);
+		if (uc->handler)
+			rc = uc->handler(uc, regs);
+
+		if (uc->rp_handler)
+			prepare_uretprobe(uprobe, regs); /* put bp at return */

This doesn't look right. prepare_uretprobe() should not be called multiple times.

Oleg.

#6 lore raw

Re: [RFC PATCH v2 3/4] uretprobes: return probe entry, prepare uretprobe

Quoted 7 lines · Oleg Nesterov
Quoted 3 lines · Anton Arapov

todo:
protect uprobe

Yep ;)

Quoted 8 lines

kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1338

Quoted 6 lines

static struct uprobe_task *get_utask(void)
{

kernel/events/uprobes.c +2 -1
-	if (!current->utask)
+	if (!current->utask) {
 		current->utask = kzalloc(sizeof(struct uprobe_task), GFP_KERNEL);
+		INIT_HLIST_HEAD(&current->utask->return_uprobes);

What if kzalloc() fails?

:)

Quoted 16 lines

kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1354

Quoted 13 lines
kernel/events/uprobes.c +12 -0
+static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)
+{
+	struct return_uprobe_i *ri;
+	struct uprobe_task *utask;
+	struct xol_area *area;
+	unsigned long rp_trampoline_vaddr = 0;
+
+	area = get_xol_area();
+	if (area)
+		rp_trampoline_vaddr = area->rp_trampoline_vaddr;
+	if (!rp_trampoline_vaddr) {
+		rp_trampoline_vaddr = xol_get_trampoline_slot();

This is obviously racy. But again, so far I think we can simply remove this.

working on this...

Quoted 6 lines
Quoted 6 lines

static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) {

kernel/events/uprobes.c +1 -0
+	int rc = 0;
 	struct uprobe_consumer *uc;
 	int remove = UPROBE_HANDLER_REMOVE;
Quoted 11 lines

kernel/events/uprobes.c static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp) · line 1537

Quoted 8 lines
kernel/events/uprobes.c +5 -1
 	down_read(&uprobe->register_rwsem);
 	for (uc = uprobe->consumers; uc; uc = uc->next) {
-		int rc = uc->handler(uc, regs);
+		if (uc->handler)
+			rc = uc->handler(uc, regs);
+
+		if (uc->rp_handler)
+			prepare_uretprobe(uprobe, regs); /* put bp at return */

This doesn't look right. prepare_uretprobe() should not be called multiple times.

Overlooked this issue, thanks for catching.

Anton.

Quoted 3 lines

Oleg.

#7 lore raw

RFC PATCH v2 4/4 uretprobes: invoke return probe handlers

In reply to Anton Arapov

uretprobe handlers are invoked when the trampoline is hit, on completion the trampoline is replaced with the saved return address and the uretprobe instance deleted.

v2:
get rid of ->return_consumers
use rp_handler()

---
 arch/x86/include/asm/uprobes.h |  5 ++++
 include/linux/uprobes.h        |  4 +++
 kernel/events/uprobes.c        | 61 +++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 69 insertions(+), 1 deletion(-)
arch/x86/include/asm/uprobes.h +5 -0
index c353555..9a605b8 100644
@@ -56,4 +56,9 @@ extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);
 extern int  arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data);
 extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
 extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long rp_trampoline_vaddr, struct pt_regs *regs);
+
+static inline unsigned long arch_uretprobe_get_sp(struct pt_regs *regs)
+{
+	return (unsigned long)regs->sp;
+}
 #endif	/* _ASM_UPROBES_H */
include/linux/uprobes.h +4 -0
index cdc4d53..1a229db 100644
@@ -45,7 +45,11 @@ enum uprobe_filter_ctx {
 };
 
 struct uprobe_consumer {
+	/* probe handler */
 	int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
+	/* return probe handler */
+	int (*rp_handler)(struct uprobe_consumer *self, struct pt_regs *regs);
+
 	bool (*filter)(struct uprobe_consumer *self,
 				enum uprobe_filter_ctx ctx,
 				struct mm_struct *mm);
kernel/events/uprobes.c +60 -1

78 lines not on this page · raw message

Signature · 1 line
1.8.0.2
maintainer
#8 lore raw

Re: [RFC PATCH v2 4/4] uretprobes: invoke return probe handlers

kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1614

Quoted 11 lines · Anton Arapov

static void handle_swbp(struct pt_regs *regs)
{

kernel/events/uprobes.c +2 -1
 	struct uprobe *uprobe;
+	struct xol_area *area;
 	unsigned long bp_vaddr;
 	int uninitialized_var(is_swbp);
 
 	bp_vaddr = uprobe_get_swbp_addr(regs);
-	uprobe = find_active_uprobe(bp_vaddr, &is_swbp);
+	area = get_xol_area();

No, we do not need to allocate xol area here.

kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1617

Quoted 3 lines
kernel/events/uprobes.c +3 -0
+	if (area) {
+		if (bp_vaddr == area->vaddr) {
+			handle_rp_swbp(regs);

Can't understand... this should check bp_vaddr == rp_trampoline_vaddr ?

Again, unless you remove rp_trampoline_vaddr altogether.

Oleg.

#9 lore raw

Re: [RFC PATCH v2 4/4] uretprobes: invoke return probe handlers

Quoted 15 lines · Oleg Nesterov

kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1614

Quoted 11 lines · Anton Arapov

static void handle_swbp(struct pt_regs *regs)
{

kernel/events/uprobes.c +2 -1
 	struct uprobe *uprobe;
+	struct xol_area *area;
 	unsigned long bp_vaddr;
 	int uninitialized_var(is_swbp);
 
 	bp_vaddr = uprobe_get_swbp_addr(regs);
-	uprobe = find_active_uprobe(bp_vaddr, &is_swbp);
+	area = get_xol_area();

No, we do not need to allocate xol area here.

Quoted 7 lines

kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1617

Quoted 3 lines
kernel/events/uprobes.c +3 -0
+	if (area) {
+		if (bp_vaddr == area->vaddr) {
+			handle_rp_swbp(regs);

Can't understand... this should check bp_vaddr == rp_trampoline_vaddr ?

Again, unless you remove rp_trampoline_vaddr altogether.

yeah, forgot to correct this chunk before I sent it here. I was working on xol_get_trampoline_slot removal as you suggested.

thanks for valuable comments, seems you are okay with approach to add rp_handler into uprobe_consumer struct.

Anton.

maintainer
#10 lore raw
In reply to Anton Arapov
Quoted 7 lines · Anton Arapov

There are RFC uretprobes implementation. I'd be grateful for review.

RFCv1: https://lkml.org/lkml/2012/12/21/133

I've intentionally removed the retprobe bypass logic, it requires a bit more work.

Yes, this is not trivial, lets do this separately.

Quoted 2 lines

not fixed since last prior RFC review:
unify xol_get_trampoline_slot() and xol_take_insn_slot()

This was of the reasons for "Do not play with utask in xol_get_insn_slot()" I sent. After this patch you only need the trivial change

- static unsigned long xol_get_insn_slot(struct uprobe *uprobe)
+ static unsigned long xol_get_insn_slot(unsigned char *insn)

and now you do not need xol_get_trampoline_slot().

However. Why do you need it at all? Let me quote myself:

	Or. Perhaps even better, do not add this helper at all. xol_alloc_area()
	could reserve the first slot/bit for trampoline. And note that in this
	case we do not need xol_area->rp_trampoline_vaddr, it is always equal
	to xol_area->vaddr.

?

Quoted 1 line

protect uprobe in prepare_uretprobe()

This should be fixed ;)

Oleg.