uprobes: return probe implementation
Subsystems (3)
v2 cover uprobes: return probe implementation
There are RFC uretprobes implementation. I'd be grateful for review.
RFCv1: https://lkml.org/lkml/2012/12/21/133
I've intentionally removed the retprobe bypass logic, it requires a bit more work.
not fixed since last prior RFC review:
unify xol_get_trampoline_slot() and xol_take_insn_slot() protect uprobe in prepare_uretprobe()
v2 changes:
introduced rp_handler(), get rid of return_consumers get rid of uretprobe_[un]register() introduced arch_uretprobe_get_sp()
removed uprobe_task->doomed, kill task immediately fix arch_uretprobe_hijack_return_addr()'s returns address the v1 minor issues
thanks,
Anton Arapov (5):
uretprobes/x86: hijack return address
uretprobes: trampoline implementation
uretprobes: return probe entry, prepare uretprobe
uretprobes: invoke return probe handlers
arch/x86/include/asm/uprobes.h | 6 ++ arch/x86/kernel/uprobes.c | 48 +++++++++++ include/linux/uprobes.h | 9 ++ kernel/events/uprobes.c | 181 ++++++++++++++++++++++++++++++++++++++++- 4 files changed, 241 insertions(+), 3 deletions(-)
Signature · 1 line
1.8.0.2
RFC PATCH v2 1/4 uretprobes/x86: hijack return address
Function to hijack the return address, replace it with a "trampoline" and function to predict the stack pointer address value at return.
v2:
remove ->doomed flag, kill task immediately
--- arch/x86/include/asm/uprobes.h | 1 + arch/x86/kernel/uprobes.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+)
arch/x86/include/asm/uprobes.h +1 -0
@@ -55,4 +55,5 @@ extern int arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);extern bool arch_uprobe_xol_was_trapped (struct task_struct *tsk);extern int arch_uprobe_exception_notify (struct notifier_block *self,unsigned long val,void *data);extern void arch_uprobe_abort_xol (struct arch_uprobe *aup,struct pt_regs *regs); +extern unsigned long arch_uretprobe_hijack_return_addr (unsigned long rp_trampoline_vaddr,struct pt_regs *regs);#endif /* _ASM_UPROBES_H */
arch/x86/kernel/uprobes.c +29 -0
@@ -697,3 +697,32 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)send_sig (SIGTRAP , current,0 );return ret; } + +extern unsigned long arch_uretprobe_hijack_return_addr (unsigned long + rp_trampoline_vaddr,struct pt_regs *regs) +{ +int rasize, ncopied; +unsigned long orig_ret_vaddr =0 ;/* clear high bits for 32-bit apps */ + + rasize =is_ia32_task () ?4 :8 ; + ncopied =copy_from_user (&orig_ret_vaddr, (void __user *)regs->sp, rasize); +if (unlikely (ncopied)) +return 0 ; + +/* check whether address has been already hijacked */ +if (orig_ret_vaddr == rp_trampoline_vaddr) +return orig_ret_vaddr; + + ncopied =copy_to_user ((void __user *)regs->sp, &rp_trampoline_vaddr, rasize); +if (unlikely (ncopied)) { +if (ncopied != rasize) { +printk (KERN_ERR "uretprobe: return address clobbered: " +"pid=%d, %%sp=%#lx, %%ip=%#lx\n" , + current->pid, regs->sp, regs->ip); +/* kill task immediately */ +send_sig (SIGSEGV , current,0 ); + } + } + +return orig_ret_vaddr; +}
Signature · 1 line
1.8.0.2
RFC PATCH v2 2/4 uretprobes: trampoline implementation
In reply to Anton ArapovThe first time a uprobe with return consumer is hit for a process, a trampoline slot is obtained in the xol_area and initialized with a breakpoint instruction. This slot is subsequently used by all uretprobes.
todo:
unify with xol_take_insn_slot()
--- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+)
include/linux/uprobes.h +1 -0
@@ -91,6 +91,7 @@ struct xol_area {* the vma go away, and we must handle that reasonably gracefully. */ unsigned long vaddr;/* Page(s) of instruction slots */ +unsigned long rp_trampoline_vaddr;/* address of trampolines */ };struct uprobes_state {
kernel/events/uprobes.c +33 -0
@@ -1208,6 +1208,39 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)return xol_vaddr; } +/* +* xol_get_trampoline_slot - A trampoline slot is obtained the first +* time a uprobe with return consumer is hit for a process. Use one +* trampoline slot for all probes pertaining to a process, i.e. +* one per xol_area. +*/ +static unsigned long xol_get_trampoline_slot (void ) +{ +struct xol_area *area; +unsigned long offset; +uprobe_opcode_t bp_insn =UPROBE_SWBP_INSN ; +void *vaddr; + + area =get_xol_area (); +if (!area) +return 0 ; + +if (!area->rp_trampoline_vaddr) + area->rp_trampoline_vaddr =xol_take_insn_slot (area); + +if (unlikely (!area->rp_trampoline_vaddr)) +return 0 ; + + offset = area->rp_trampoline_vaddr & ~PAGE_MASK ; + vaddr =kmap_atomic (area->page); +memcpy (vaddr + offset, &bp_insn,UPROBE_SWBP_INSN_SIZE ); +kunmap_atomic (vaddr); + +flush_dcache_page (area->page); + +return area->rp_trampoline_vaddr; +} +/* * xol_free_insn_slot - If slot was earlier allocated by * @xol_get_insn_slot(), make the slot available for
Signature · 1 line
1.8.0.2
Oleg Nesterov
maintainerRe: [RFC PATCH v2 2/4] uretprobes: trampoline implementation
Quoted 5 lines · Anton Arapov
The first time a uprobe with return consumer is hit for a process, a trampoline slot is obtained in the
xol_areaand initialized with a breakpoint instruction. This slot is subsequently used by all uretprobes.
See my reply to 0/4, I think we do not really need this.
Quoted 68 lines
todo:
unify withxol_take_insn_slot()Signed-off-by Anton Arapov <[email protected]>--- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+)include/linux/uprobes.h +1
-0@@ -91,6 +91,7 @@ struct xol_area {* the vma go away, and we must handle that reasonably gracefully. */ unsigned long vaddr;/* Page(s) of instruction slots */ +unsigned long rp_trampoline_vaddr;/* address of trampolines */ };struct uprobes_state {kernel/events/uprobes.c +33
-0@@ -1208,6 +1208,39 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)return xol_vaddr; } +/* +* xol_get_trampoline_slot - A trampoline slot is obtained the first +* time a uprobe with return consumer is hit for a process. Use one +* trampoline slot for all probes pertaining to a process, i.e. +* one per xol_area. +*/ +static unsigned long xol_get_trampoline_slot (void ) +{ +struct xol_area *area; +unsigned long offset; +uprobe_opcode_t bp_insn =UPROBE_SWBP_INSN ; +void *vaddr; + + area =get_xol_area (); +if (!area) +return 0 ; + +if (!area->rp_trampoline_vaddr) + area->rp_trampoline_vaddr =xol_take_insn_slot (area); + +if (unlikely (!area->rp_trampoline_vaddr)) +return 0 ; + + offset = area->rp_trampoline_vaddr & ~PAGE_MASK ; + vaddr =kmap_atomic (area->page); +memcpy (vaddr + offset, &bp_insn,UPROBE_SWBP_INSN_SIZE ); +kunmap_atomic (vaddr); + +flush_dcache_page (area->page); + +return area->rp_trampoline_vaddr; +} +/* * xol_free_insn_slot - If slot was earlier allocated by * @xol_get_insn_slot(), make the slot available for Signature · 1 line
1.8.0.2
RFC PATCH v2 3/4 uretprobes: return probe entry, prepare uretprobe
In reply to Anton ArapovWhen a uprobe with return consumer is hit, prepare_uretprobe function is invoked. It creates return_instance, hijacks return address and replaces it with the trampoline.
v2:
get rid of ->return_consumers
todo:
protect uprobe
--- include/linux/uprobes.h | 4 ++++ kernel/events/uprobes.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 50 insertions(+), 2 deletions(-)
include/linux/uprobes.h +4 -0
@@ -68,6 +68,10 @@ struct uprobe_task {enum uprobe_task_state state;struct arch_uprobe_task autask; +/* +* list for tracking uprobes with return consumers +*/ +struct hlist_head return_uprobes;struct uprobe *active_uprobe;unsigned long xol_vaddr;
kernel/events/uprobes.c +46 -2
75 lines not on this page · raw message
Signature · 1 line
1.8.0.2
Oleg Nesterov
maintainerRe: [RFC PATCH v2 3/4] uretprobes: return probe entry, prepare uretprobe
Quoted 3 lines · Anton Arapov
todo:
protect uprobe
Yep ;)
kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1338
Quoted 6 lines
static struct
uprobe_task*get_utask(void)
{kernel/events/uprobes.c +2
-1-if (!current->utask) +if (!current->utask) { current->utask =kzalloc (sizeof (struct uprobe_task ),GFP_KERNEL ); +INIT_HLIST_HEAD (¤t->utask->return_uprobes);
What if kzalloc() fails?
kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1354
Quoted 12 lines
kernel/events/uprobes.c +12
-0+static void prepare_uretprobe (struct uprobe *uprobe,struct pt_regs *regs) +{ +struct return_uprobe_i *ri; +struct uprobe_task *utask; +struct xol_area *area; +unsigned long rp_trampoline_vaddr =0 ; + + area =get_xol_area (); +if (area) + rp_trampoline_vaddr = area->rp_trampoline_vaddr; +if (!rp_trampoline_vaddr) { + rp_trampoline_vaddr =xol_get_trampoline_slot ();
This is obviously racy. But again, so far I think we can simply remove this.
Quoted 4 lines
static void
handler_chain(struct uprobe *uprobe, struct pt_regs *regs){kernel/events/uprobes.c +1
-0+int rc =0 ;struct uprobe_consumer *uc;
kernel/events/uprobes.c static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp) · line 1537
Quoted 10 lines
kernel/events/uprobes.c +5
-1int remove =UPROBE_HANDLER_REMOVE ;down_read (&uprobe->register_rwsem);for (uc = uprobe->consumers; uc; uc = uc->next) { -int rc = uc->handler (uc, regs); +if (uc->handler) + rc = uc->handler (uc, regs); + +if (uc->rp_handler) +prepare_uretprobe (uprobe, regs);/* put bp at return */
This doesn't look right. prepare_uretprobe() should not be called multiple times.
Oleg.
Re: [RFC PATCH v2 3/4] uretprobes: return probe entry, prepare uretprobe
Quoted 7 lines · Oleg Nesterov
Quoted 3 lines · Anton Arapov
todo:
protect uprobeYep ;)
Quoted 8 lines
kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1338
Quoted 6 lines
static struct
uprobe_task*get_utask(void)
{kernel/events/uprobes.c +2
-1-if (!current->utask) +if (!current->utask) { current->utask =kzalloc (sizeof (struct uprobe_task ),GFP_KERNEL ); +INIT_HLIST_HEAD (¤t->utask->return_uprobes);What if
kzalloc()fails?
:)
Quoted 16 lines
kernel/events/uprobes.c void uprobe_copy_process(struct task_struct *t) · line 1354
Quoted 13 lines
kernel/events/uprobes.c +12
-0+static void prepare_uretprobe (struct uprobe *uprobe,struct pt_regs *regs) +{ +struct return_uprobe_i *ri; +struct uprobe_task *utask; +struct xol_area *area; +unsigned long rp_trampoline_vaddr =0 ; + + area =get_xol_area (); +if (area) + rp_trampoline_vaddr = area->rp_trampoline_vaddr; +if (!rp_trampoline_vaddr) { + rp_trampoline_vaddr =xol_get_trampoline_slot ();This is obviously racy. But again, so far I think we can simply remove this.
working on this...
Quoted 6 lines
Quoted 6 lines
static void
handler_chain(struct uprobe *uprobe, struct pt_regs *regs){kernel/events/uprobes.c +1
-0+int rc =0 ;struct uprobe_consumer *uc;int remove =UPROBE_HANDLER_REMOVE ;
Quoted 11 lines
kernel/events/uprobes.c static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp) · line 1537
Quoted 8 lines
kernel/events/uprobes.c +5
-1down_read (&uprobe->register_rwsem);for (uc = uprobe->consumers; uc; uc = uc->next) { -int rc = uc->handler (uc, regs); +if (uc->handler) + rc = uc->handler (uc, regs); + +if (uc->rp_handler) +prepare_uretprobe (uprobe, regs);/* put bp at return */ This doesn't look right.
prepare_uretprobe()should not be called multiple times.
Overlooked this issue, thanks for catching.
Anton.
Quoted 3 lines
Oleg.
RFC PATCH v2 4/4 uretprobes: invoke return probe handlers
In reply to Anton Arapovuretprobe handlers are invoked when the trampoline is hit, on completion the trampoline is replaced with the saved return address and the uretprobe instance deleted.
v2:
get rid of ->return_consumers
use rp_handler()
--- arch/x86/include/asm/uprobes.h | 5 ++++ include/linux/uprobes.h | 4 +++ kernel/events/uprobes.c | 61 +++++++++++++++++++++++++++++++++++++++++- 3 files changed, 69 insertions(+), 1 deletion(-)
arch/x86/include/asm/uprobes.h +5 -0
@@ -56,4 +56,9 @@ extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);extern int arch_uprobe_exception_notify (struct notifier_block *self,unsigned long val,void *data);extern void arch_uprobe_abort_xol (struct arch_uprobe *aup,struct pt_regs *regs);extern unsigned long arch_uretprobe_hijack_return_addr (unsigned long rp_trampoline_vaddr,struct pt_regs *regs); + +static inline unsigned long arch_uretprobe_get_sp (struct pt_regs *regs) +{ +return (unsigned long )regs->sp; +}#endif /* _ASM_UPROBES_H */
include/linux/uprobes.h +4 -0
@@ -45,7 +45,11 @@ enum uprobe_filter_ctx { };struct uprobe_consumer { +/* probe handler */ int (*handler)(struct uprobe_consumer *self,struct pt_regs *regs); +/* return probe handler */ +int (*rp_handler)(struct uprobe_consumer *self,struct pt_regs *regs); +bool (*filter)(struct uprobe_consumer *self,enum uprobe_filter_ctx ctx,struct mm_struct *mm);
kernel/events/uprobes.c +60 -1
78 lines not on this page · raw message
Signature · 1 line
1.8.0.2
Oleg Nesterov
maintainerRe: [RFC PATCH v2 4/4] uretprobes: invoke return probe handlers
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1614
Quoted 11 lines · Anton Arapov
static void
handle_swbp(struct pt_regs *regs)
{kernel/events/uprobes.c +2
-1struct uprobe *uprobe; +struct xol_area *area;unsigned long bp_vaddr;int uninitialized_var (is_swbp ); bp_vaddr =uprobe_get_swbp_addr (regs); - uprobe =find_active_uprobe (bp_vaddr, &is_swbp); + area =get_xol_area ();
No, we do not need to allocate xol area here.
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1617
Quoted 3 lines
kernel/events/uprobes.c +3
-0+if (area) { +if (bp_vaddr == area->vaddr) { +handle_rp_swbp (regs);
Can't understand... this should check bp_vaddr == rp_trampoline_vaddr ?
Again, unless you remove rp_trampoline_vaddr altogether.
Oleg.
Re: [RFC PATCH v2 4/4] uretprobes: invoke return probe handlers
Quoted 15 lines · Oleg Nesterov
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1614
Quoted 11 lines · Anton Arapov
static void
handle_swbp(struct pt_regs *regs)
{kernel/events/uprobes.c +2
-1struct uprobe *uprobe; +struct xol_area *area;unsigned long bp_vaddr;int uninitialized_var (is_swbp ); bp_vaddr =uprobe_get_swbp_addr (regs); - uprobe =find_active_uprobe (bp_vaddr, &is_swbp); + area =get_xol_area ();No, we do not need to allocate xol area here.
Quoted 7 lines
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1617
Quoted 3 lines
kernel/events/uprobes.c +3
-0+if (area) { +if (bp_vaddr == area->vaddr) { +handle_rp_swbp (regs);Can't understand... this should check
bp_vaddr==rp_trampoline_vaddr?Again, unless you remove
rp_trampoline_vaddraltogether.
yeah, forgot to correct this chunk before I sent it here. I was working on xol_get_trampoline_slot removal as you suggested.
thanks for valuable comments, seems you are okay with approach to add rp_handler into uprobe_consumer struct.
Anton.
Oleg Nesterov
maintainerQuoted 7 lines · Anton Arapov
There are RFC uretprobes implementation. I'd be grateful for review.
RFCv1: https://lkml.org/lkml/2012/12/21/133
I've intentionally removed the retprobe bypass logic, it requires a bit more work.
Yes, this is not trivial, lets do this separately.
Quoted 2 lines
not fixed since last prior RFC review:
unifyxol_get_trampoline_slot()andxol_take_insn_slot()
This was of the reasons for "Do not play with utask in xol_get_insn_slot()" I sent. After this patch you only need the trivial change
- static unsigned long xol_get_insn_slot(struct uprobe *uprobe)
+ static unsigned long xol_get_insn_slot(unsigned char *insn)
and now you do not need xol_get_trampoline_slot().
However. Why do you need it at all? Let me quote myself:
Or. Perhaps even better, do not add this helper at all. xol_alloc_area() could reserve the first slot/bit for trampoline. And note that in this case we do not need xol_area->rp_trampoline_vaddr, it is always equal to xol_area->vaddr.
?
Quoted 1 line
protect uprobe in
prepare_uretprobe()
This should be fixed ;)
Oleg.