uretprobes: return probes implementation
Subsystems (3)
cover uretprobes: return probes implementation
Hello All,
this is core implementation of the uretprobes. This enables a breakpoint
on function's return for the tools such as perf.
Introduce additional handler* for uprobe consumer that makes possible the
distinguish uprobe from uretprobe. Note, behind every uretprobe a regular uprobe with return probe handler(rp_handler). Once hit the uprobe that has rp_handler, we hijack the return address of the probed function and replacing it with the address of trampoline. Trampoline is the preallocated page in probed task's xol area that filled with breakpoint opcode. In turn, when the return breakpoint is hit, invoke the rp_handler.
The patchset shouldn't be difficult to read and hopefully the comments to
commits will help.
Please, review.
patchset in git:
http://github.com/arapov/linux-aa/commits/uretprobes_v0
RFC reviews:
v4: https://lkml.org/lkml/2013/3/4/246
v3: https://lkml.org/lkml/2013/2/28/148
v2: https://lkml.org/lkml/2013/1/9/157
v1: https://lkml.org/lkml/2012/12/21/133
thanks,
Anton.
Anton Arapov (7):
uretprobes: preparation patch
uretprobes: extract fill_page() and trampoline implementation uretprobes/x86: hijack return address
uretprobes: return probe entry, prepare_uretprobe()
uretprobes: return probe exit, invoke handlers
uretprobes: limit the depth of return probe nestedness
uretprobes: remove -ENOSYS as return probes implemented
arch/x86/include/asm/uprobes.h | 1 + arch/x86/kernel/uprobes.c | 29 ++++++ include/linux/uprobes.h | 5 + kernel/events/uprobes.c | 205 ++++++++++++++++++++++++++++++++++++++--- 4 files changed, 229 insertions(+), 11 deletions(-)
Signature · 1 line
1.8.1.4
PATCH 1/7 uretprobes: preparation patch
Enclose return probes implementation, introduce ->rp_handler() and update
existing code to rely on ->handler() *and* ->rp_handler() for uprobe and uretprobe respectively.
RFCv5 changes:
- don't remove uprobe in case there are no uprobe consumer(handler), see handler_chain() changes.
RFCv3 changes: (the patch is introduced in v3)
- check whether at least one of the consumer's handlers were set.
- a 'TODO' cap that will be removed once return probes be implemented.
- introduce ->rp_handler().
--- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 14 +++++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-)
include/linux/uprobes.h +1 -0
@@ -46,6 +46,7 @@ enum uprobe_filter_ctx {struct uprobe_consumer {int (*handler)(struct uprobe_consumer *self,struct pt_regs *regs); +int (*rp_handler)(struct uprobe_consumer *self,struct pt_regs *regs);bool (*filter)(struct uprobe_consumer *self,enum uprobe_filter_ctx ctx,struct mm_struct *mm);
kernel/events/uprobes.c +13 -1
@@ -828,6 +828,14 @@ int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *struct uprobe *uprobe;int ret; +/* Uprobe must have at least one set consumer */ +if (!uc->handler && !uc->rp_handler) +return -EINVAL ; + +/* TODO: Implement return probes */ +if (uc->rp_handler) +return -ENOSYS ; +/* Racy, just to catch the obvious mistakes */ if (offset >i_size_read (inode))return -EINVAL ;
@@ -1488,10 +1496,14 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) {struct uprobe_consumer *uc;int remove =UPROBE_HANDLER_REMOVE ; +int rc =0 ;down_read (&uprobe->register_rwsem);for (uc = uprobe->consumers; uc; uc = uc->next) { -int rc = uc->handler (uc, regs); +if (uc->handler) + rc = uc->handler (uc, regs); +else + remove =0 ;WARN (rc & ~UPROBE_HANDLER_MASK ,"bad rc=0x%x from %pf()\n" , rc, uc->handler);
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerRe: [PATCH 1/7] uretprobes: preparation patch
Quoted 4 lines · Anton Arapov
(quoted diff) +0
-0@@ -1488,10 +1496,14 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) { struct uprobe_consumer *uc;
Review static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1506
Quoted 10 lines
(quoted diff) +5
-1int remove = UPROBE_HANDLER_REMOVE; + int rc = 0; down_read(&uprobe->register_rwsem); for (uc = uprobe->consumers; uc; uc = uc->next) { - int rc = uc->handler(uc, regs); + if (uc->handler) + rc = uc->handler(uc, regs); + else + remove = 0;
Well, this doesn't look good. Yes, we need to conditionalize uc->handler() and rc checks, but the code looks ugly. We touch remove twice, and the value of rc inside the loop is bogus if ->handler == NULL.
I wouldn't have argued, but, but 4/7 changes the "else" branch and this change is wrong (I'll write another email). We do not need this "else" at all.
I'd suggest the patch below.
Oleg.
kernel/events/uprobes.c +6 -3
@@ -1491,10 +1491,13 @@ static void handler_chain(struct uprobedown_read (&uprobe->register_rwsem);for (uc = uprobe->consumers; uc; uc = uc->next) { -int rc = uc->handler (uc, regs); +int rc =0 ; -WARN (rc & ~UPROBE_HANDLER_MASK , -"bad rc=0x%x from %pf()\n" , rc, uc->handler); +if (uc->handler) { + rc = uc->handler (uc, regs); +WARN (rc & ~UPROBE_HANDLER_MASK , +"bad rc=0x%x from %pf()\n" , rc, uc->handler); + } remove &= rc; }
PATCH 2/7 uretprobes: extract fill_page() and trampoline implementation
In reply to Anton Arapov - Split out fill_page() from xol_get_insn_slot().
- Allocate trampoline page, as the very first one in uprobed task xol area, and fill it with breakpoint opcode.
- Also introduce get_trampoline_vaddr() helper, to wrap the
trampoline address extraction from area->vaddr. That removes
confusion and eases the debug experience in case ->vaddr
notion will be changed.--- kernel/events/uprobes.c | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-)
kernel/events/uprobes.c +26 -9
@@ -1080,6 +1080,21 @@ void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned lonset_bit (MMF_RECALC_UPROBES , &vma->vm_mm->flags); } +static void fill_page (struct page *page,unsigned long offset,uprobe_opcode_t *insn) +{ +void *vaddr; + + vaddr =kmap_atomic (page); +memcpy (vaddr + offset, insn,MAX_UINSN_BYTES ); +kunmap_atomic (vaddr); + +/* +* We probably need flush_icache_user_range() but it needs vma. +* This should work on supported architectures too. +*/ +flush_dcache_page (page); +} +/* Slot allocation for XOL */ static int xol_add_vma (struct xol_area *area) {
@@ -1122,6 +1137,7 @@ static struct xol_area *get_xol_area(void) {struct mm_struct *mm = current->mm;struct xol_area *area; +uprobe_opcode_t insn =UPROBE_SWBP_INSN ; area = mm->uprobes_state.xol_area;if (area)
@@ -1139,6 +1155,10 @@ static struct xol_area *get_xol_area(void)if (!area->page) goto free_bitmap; +/* pre-allocate for return probes */ +set_bit (0 , area->bitmap); +fill_page (area->page,0 , &insn); +init_waitqueue_head (&area->wq);if (!xol_add_vma (area))return area;
@@ -1226,7 +1246,6 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)struct xol_area *area;unsigned long offset;unsigned long xol_vaddr; -void *vaddr; area =get_xol_area ();if (!area)
@@ -1238,14 +1257,7 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)/* Initialize the slot */ offset = xol_vaddr & ~PAGE_MASK ; - vaddr =kmap_atomic (area->page); -memcpy (vaddr + offset, uprobe->arch.insn,MAX_UINSN_BYTES ); -kunmap_atomic (vaddr); -/* -* We probably need flush_icache_user_range() but it needs vma. -* This should work on supported architectures too. -*/ -flush_dcache_page (area->page); +fill_page (area->page, offset, uprobe->arch.insn);return xol_vaddr; }
@@ -1341,6 +1353,11 @@ static struct uprobe_task *get_utask(void)return current->utask; } +static unsigned long get_trampoline_vaddr (struct xol_area *area) +{ +return area->vaddr; +} +/* Prepare to single-step probed instruction out of line. */ static int pre_ssout (struct uprobe *uprobe,struct pt_regs *regs,unsigned long bp_vaddr)
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerRe: [PATCH 2/7] uretprobes: extract fill_page() and trampoline implementation
Quoted 2 lines · Anton Arapov
+static void
fill_page(struct page *page, unsigned long offset, uprobe_opcode_t *insn)
^^^^^^^^^^^^^^^
Well, it should be "u8 *insn" or "char *".
xol_get_insn_slot() passes arch.insn, and this is not uprobe_opcode_t in general (see arch/powerpc/include/asm/uprobes.h)
kernel/events/uprobes.c void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned lon · line 1088
Quoted 5 lines
kernel/events/uprobes.c +5
-0+{ +void *vaddr; + + vaddr =kmap_atomic (page); +memcpy (vaddr + offset, insn,MAX_UINSN_BYTES );
^^^^^^^^^^^^^^^
OK, but see below...
kernel/events/uprobes.c void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned lon · line 1096
Quoted 8 lines
kernel/events/uprobes.c +8
-0+kunmap_atomic (vaddr); + +/* +* We probably need flush_icache_user_range() but it needs vma. +* This should work on supported architectures too. +*/ +flush_dcache_page (page); +}
Please do not move flush_dcache_page() from xol_get_insn_slot(), this is not needed.
Unlike xol_get_insn_slot(), get_xol_area() calls fill_page() before xol_add_vma().
Quoted 11 lines
static int
xol_add_vma(struct xol_area *area)
{kernel/events/uprobes.c +1
-0@@ -1122,6 +1137,7 @@ static struct xol_area *get_xol_area(void) {struct mm_struct *mm = current->mm;struct xol_area *area; +uprobe_opcode_t insn =UPROBE_SWBP_INSN ; area = mm->uprobes_state.xol_area;if (area)
kernel/events/uprobes.c static struct xol_area *get_xol_area(void) · line 1160
Quoted 6 lines
kernel/events/uprobes.c +3
-0@@ -1139,6 +1155,10 @@ static struct xol_area *get_xol_area(void)if (!area->page) goto free_bitmap; +/* pre-allocate for return probes */ +set_bit (0 , area->bitmap); +fill_page (area->page,0 , &insn);
sizeof(insn) == UPROBE_SWBP_INSN_SIZE != MAX_UINSN_BYTES. See also the comments above.
Oleg.
Oleg Nesterov
maintainercover kmap cleanups for uretprobes (Was: extract fill_page() and trampoline implementation)
Quoted 6 lines · Oleg Nesterov
Quoted 2 lines · Anton Arapov
+static void
fill_page(struct page *page, unsigned long offset, uprobe_opcode_t *insn)^^^^^^^^^^^^^^^Well, it should be "u8 *insn" or "char *".
Or void*.
And we have another reason for this helper.
And I really think that we need to cleanup the kmap mess in uprobe.c before we add the new abuser.
How about this simple series?
Quoted 2 lines
sizeof(insn) ==
UPROBE_SWBP_INSN_SIZE!=MAX_UINSN_BYTES. See also the comments above.
I think that copy_to_page() added by 4/5 is what you need, and it solves the problems with typeof/sizeof. Plus it have another caller.
Oleg.
Oleg Nesterov
maintainerPATCH 1/5 uprobes: Turn copy_opcode() into copy_from_page()
No functional changes. Rename copy_opcode() into copy_from_page() and add the new "int len" argument to make it more more generic for the new users.
--- kernel/events/uprobes.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
kernel/events/uprobes.c +4 -4
@@ -173,10 +173,10 @@ bool __weak is_swbp_insn(uprobe_opcode_t *insn)return *insn ==UPROBE_SWBP_INSN ; } -static void copy_opcode (struct page *page,unsigned long vaddr,uprobe_opcode_t *opcode) +static void copy_from_page (struct page *page,unsigned long vaddr,void *dst,int len) {void *kaddr =kmap_atomic (page); -memcpy (opcode, kaddr + (vaddr & ~PAGE_MASK ),UPROBE_SWBP_INSN_SIZE ); +memcpy (dst, kaddr + (vaddr & ~PAGE_MASK ), len);kunmap_atomic (kaddr); }
@@ -185,7 +185,7 @@ static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_opcode_tuprobe_opcode_t old_opcode;bool is_swbp; -copy_opcode (page, vaddr, &old_opcode); +copy_from_page (page, vaddr, &old_opcode,UPROBE_SWBP_INSN_SIZE ); is_swbp =is_swbp_insn (&old_opcode);if (is_swbp_insn (new_opcode)) {
@@ -1449,7 +1449,7 @@ static int is_swbp_at_addr(struct mm_struct *mm, unsigned long vaddr)if (result <0 )return result; -copy_opcode (page, vaddr, &opcode); +copy_from_page (page, vaddr, &opcode,UPROBE_SWBP_INSN_SIZE );put_page (page); out:return is_swbp_insn (&opcode);
Signature · 1 line
1.5.5.1
Re: [PATCH 1/5] uprobes: Turn copy_opcode() into copy_from_page()
Quoted 47 lines · Oleg Nesterov
No functional changes. Rename
copy_opcode()intocopy_from_page()and add the new "int len" argument to make it more more generic for the new users.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/events/uprobes.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)kernel/events/uprobes.c +4
-4@@ -173,10 +173,10 @@ bool __weak is_swbp_insn(uprobe_opcode_t *insn)return *insn ==UPROBE_SWBP_INSN ; } -static void copy_opcode (struct page *page,unsigned long vaddr,uprobe_opcode_t *opcode) +static void copy_from_page (struct page *page,unsigned long vaddr,void *dst,int len) {void *kaddr =kmap_atomic (page); -memcpy (opcode, kaddr + (vaddr & ~PAGE_MASK ),UPROBE_SWBP_INSN_SIZE ); +memcpy (dst, kaddr + (vaddr & ~PAGE_MASK ), len);kunmap_atomic (kaddr); }@@ -185,7 +185,7 @@ static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_opcode_tuprobe_opcode_t old_opcode;bool is_swbp; -copy_opcode (page, vaddr, &old_opcode); +copy_from_page (page, vaddr, &old_opcode,UPROBE_SWBP_INSN_SIZE ); is_swbp =is_swbp_insn (&old_opcode);if (is_swbp_insn (new_opcode)) {@@ -1449,7 +1449,7 @@ static int is_swbp_at_addr(struct mm_struct *mm, unsigned long vaddr)if (result <0 )return result; -copy_opcode (page, vaddr, &opcode); +copy_from_page (page, vaddr, &opcode,UPROBE_SWBP_INSN_SIZE );put_page (page); out:return is_swbp_insn (&opcode);Signature · 1 line
1.5.5.1
Re: [PATCH 1/5] uprobes: Turn copy_opcode() into copy_from_page()
In reply to Oleg NesterovQuoted 5 lines · Oleg Nesterov
No functional changes. Rename
copy_opcode()intocopy_from_page()and add the new "int len" argument to make it more more generic for the new users.Signed-off-by Oleg Nesterov <[email protected]>
Quoted 42 lines
--- kernel/events/uprobes.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)kernel/events/uprobes.c +4
-4@@ -173,10 +173,10 @@ bool __weak is_swbp_insn(uprobe_opcode_t *insn)return *insn ==UPROBE_SWBP_INSN ; } -static void copy_opcode (struct page *page,unsigned long vaddr,uprobe_opcode_t *opcode) +static void copy_from_page (struct page *page,unsigned long vaddr,void *dst,int len) {void *kaddr =kmap_atomic (page); -memcpy (opcode, kaddr + (vaddr & ~PAGE_MASK ),UPROBE_SWBP_INSN_SIZE ); +memcpy (dst, kaddr + (vaddr & ~PAGE_MASK ), len);kunmap_atomic (kaddr); }@@ -185,7 +185,7 @@ static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_opcode_tuprobe_opcode_t old_opcode;bool is_swbp; -copy_opcode (page, vaddr, &old_opcode); +copy_from_page (page, vaddr, &old_opcode,UPROBE_SWBP_INSN_SIZE ); is_swbp =is_swbp_insn (&old_opcode);if (is_swbp_insn (new_opcode)) {@@ -1449,7 +1449,7 @@ static int is_swbp_at_addr(struct mm_struct *mm, unsigned long vaddr)if (result <0 )return result; -copy_opcode (page, vaddr, &opcode); +copy_from_page (page, vaddr, &opcode,UPROBE_SWBP_INSN_SIZE );put_page (page); out:return is_swbp_insn (&opcode);Signature · 1 line
1.5.5.1
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Oleg Nesterov
maintainerPATCH 2/5 uprobes: Change __copy_insn() to use copy_from_page()
In reply to Oleg NesterovChange __copy_insn() to use copy_from_page() and simplify the code.
--- kernel/events/uprobes.c | 13 ++----------- 1 files changed, 2 insertions(+), 11 deletions(-)
kernel/events/uprobes.c +2 -11
@@ -477,30 +477,21 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn, unsigned long nbytes, loff_t offset) {struct page *page; -void *vaddr; -unsigned long off; -pgoff_t idx;if (!filp)return -EINVAL ;if (!mapping->a_ops->readpage)return -EIO ; - - idx = offset >>PAGE_CACHE_SHIFT ; - off = offset & ~PAGE_MASK ; -/* * Ensure that the page that has the original instruction is * populated and in page-cache. */ - page =read_mapping_page (mapping, idx, filp); + page =read_mapping_page (mapping, offset >>PAGE_CACHE_SHIFT , filp);if (IS_ERR (page))return PTR_ERR (page); - vaddr =kmap_atomic (page); -memcpy (insn, vaddr + off, nbytes); -kunmap_atomic (vaddr); +copy_from_page (page, offset, insn, nbytes);page_cache_release (page);return 0 ;
Signature · 1 line
1.5.5.1
Re: [PATCH 2/5] uprobes: Change __copy_insn() to use copy_from_page()
Quoted 47 lines · Oleg Nesterov
Change
__copy_insn()to usecopy_from_page()and simplify the code.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/events/uprobes.c | 13 ++----------- 1 files changed, 2 insertions(+), 11 deletions(-)kernel/events/uprobes.c +2
-11@@ -477,30 +477,21 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn, unsigned long nbytes, loff_t offset) {struct page *page; -void *vaddr; -unsigned long off; -pgoff_t idx;if (!filp)return -EINVAL ;if (!mapping->a_ops->readpage)return -EIO ; - - idx = offset >>PAGE_CACHE_SHIFT ; - off = offset & ~PAGE_MASK ; -/* * Ensure that the page that has the original instruction is * populated and in page-cache. */ - page =read_mapping_page (mapping, idx, filp); + page =read_mapping_page (mapping, offset >>PAGE_CACHE_SHIFT , filp);if (IS_ERR (page))return PTR_ERR (page); - vaddr =kmap_atomic (page); -memcpy (insn, vaddr + off, nbytes); -kunmap_atomic (vaddr); +copy_from_page (page, offset, insn, nbytes);page_cache_release (page);return 0 ;Signature · 1 line
1.5.5.1
Re: [PATCH 2/5] uprobes: Change __copy_insn() to use copy_from_page()
In reply to Oleg NesterovQuoted 3 lines · Oleg Nesterov
Change
__copy_insn()to usecopy_from_page()and simplify the code.Signed-off-by Oleg Nesterov <[email protected]>
Quoted 44 lines
--- kernel/events/uprobes.c | 13 ++----------- 1 files changed, 2 insertions(+), 11 deletions(-)kernel/events/uprobes.c +2
-11@@ -477,30 +477,21 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn, unsigned long nbytes, loff_t offset) {struct page *page; -void *vaddr; -unsigned long off; -pgoff_t idx;if (!filp)return -EINVAL ;if (!mapping->a_ops->readpage)return -EIO ; - - idx = offset >>PAGE_CACHE_SHIFT ; - off = offset & ~PAGE_MASK ; -/* * Ensure that the page that has the original instruction is * populated and in page-cache. */ - page =read_mapping_page (mapping, idx, filp); + page =read_mapping_page (mapping, offset >>PAGE_CACHE_SHIFT , filp);if (IS_ERR (page))return PTR_ERR (page); - vaddr =kmap_atomic (page); -memcpy (insn, vaddr + off, nbytes); -kunmap_atomic (vaddr); +copy_from_page (page, offset, insn, nbytes);page_cache_release (page);return 0 ;Signature · 1 line
1.5.5.1
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Oleg Nesterov
maintainerPATCH 3/5 uprobes: Kill the unnecesary filp != NULL check in __copy_insn()
In reply to Oleg Nesterov__copy_insn(filp) can only be called after valid_vma() returns T, vma->vm_file passed as "filp" can not be NULL.
--- kernel/events/uprobes.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-)
kernel/events/uprobes.c +0 -3
@@ -478,9 +478,6 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn, {struct page *page; -if (!filp) -return -EINVAL ; -if (!mapping->a_ops->readpage)return -EIO ;/*
Signature · 1 line
1.5.5.1
Re: [PATCH 3/5] uprobes: Kill the unnecesary filp != NULL check in __copy_insn()
Quoted 25 lines · Oleg Nesterov
__copy_insn(filp)can only be called aftervalid_vma()returns T,vma->vm_filepassed as "filp" can not be NULL.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/events/uprobes.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-)kernel/events/uprobes.c +0
-3@@ -478,9 +478,6 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn, {struct page *page; -if (!filp) -return -EINVAL ; -if (!mapping->a_ops->readpage)return -EIO ;/* Signature · 1 line
1.5.5.1
Re: [PATCH 3/5] uprobes: Kill the unnecesary filp != NULL check in __copy_insn()
In reply to Oleg NesterovQuoted 4 lines · Oleg Nesterov
__copy_insn(filp)can only be called aftervalid_vma()returns T,vma->vm_filepassed as "filp" can not be NULL.Signed-off-by Oleg Nesterov <[email protected]>
Quoted 21 lines
--- kernel/events/uprobes.c | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-)kernel/events/uprobes.c +0
-3@@ -478,9 +478,6 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn, {struct page *page; -if (!filp) -return -EINVAL ; -if (!mapping->a_ops->readpage)return -EIO ;/* Signature · 1 line
1.5.5.1
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Oleg Nesterov
maintainerPATCH 4/5 uprobes: Introduce copy_to_page()
In reply to Oleg NesterovExtract the kmap_atomic/memcpy/kunmap_atomic code from xol_get_insn_slot() into the new simple helper, copy_to_page(). It will have more users soon.
--- kernel/events/uprobes.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-)
kernel/events/uprobes.c +8 -6
@@ -180,6 +180,13 @@ static void copy_from_page(struct page *page, unsigned long vaddr, void *dst, inkunmap_atomic (kaddr); } +static void copy_to_page (struct page *page,unsigned long vaddr,const void *src,int len) +{ +void *kaddr =kmap_atomic (page); +memcpy (kaddr + (vaddr & ~PAGE_MASK ), src, len); +kunmap_atomic (kaddr); +} +static int verify_opcode (struct page *page,unsigned long vaddr,uprobe_opcode_t *new_opcode) {uprobe_opcode_t old_opcode;
@@ -1204,9 +1211,7 @@ static unsigned long xol_take_insn_slot(struct xol_area *area)static unsigned long xol_get_insn_slot (struct uprobe *uprobe) {struct xol_area *area; -unsigned long offset;unsigned long xol_vaddr; -void *vaddr; area =get_xol_area ();if (!area)
@@ -1217,10 +1222,7 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)return 0 ;/* Initialize the slot */ - offset = xol_vaddr & ~PAGE_MASK ; - vaddr =kmap_atomic (area->page); -memcpy (vaddr + offset, uprobe->arch.insn,MAX_UINSN_BYTES ); -kunmap_atomic (vaddr); +copy_to_page (area->page, xol_vaddr, uprobe->arch.insn,MAX_UINSN_BYTES );/* * We probably need flush_icache_user_range() but it needs vma. * This should work on supported architectures too.
Signature · 1 line
1.5.5.1
Re: [PATCH 4/5] uprobes: Introduce copy_to_page()
Quoted 52 lines · Oleg Nesterov
Extract the kmap_atomic/memcpy/kunmap_atomic code from
xol_get_insn_slot()into the new simple helper,copy_to_page(). It will have more users soon.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/events/uprobes.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-)kernel/events/uprobes.c +8
-6@@ -180,6 +180,13 @@ static void copy_from_page(struct page *page, unsigned long vaddr, void *dst, inkunmap_atomic (kaddr); } +static void copy_to_page (struct page *page,unsigned long vaddr,const void *src,int len) +{ +void *kaddr =kmap_atomic (page); +memcpy (kaddr + (vaddr & ~PAGE_MASK ), src, len); +kunmap_atomic (kaddr); +} +static int verify_opcode (struct page *page,unsigned long vaddr,uprobe_opcode_t *new_opcode) {uprobe_opcode_t old_opcode;@@ -1204,9 +1211,7 @@ static unsigned long xol_take_insn_slot(struct xol_area *area)static unsigned long xol_get_insn_slot (struct uprobe *uprobe) {struct xol_area *area; -unsigned long offset;unsigned long xol_vaddr; -void *vaddr; area =get_xol_area ();if (!area)@@ -1217,10 +1222,7 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)return 0 ;/* Initialize the slot */ - offset = xol_vaddr & ~PAGE_MASK ; - vaddr =kmap_atomic (area->page); -memcpy (vaddr + offset, uprobe->arch.insn,MAX_UINSN_BYTES ); -kunmap_atomic (vaddr); +copy_to_page (area->page, xol_vaddr, uprobe->arch.insn,MAX_UINSN_BYTES );/* * We probably need flush_icache_user_range() but it needs vma. * This should work on supported architectures too. Signature · 1 line
1.5.5.1
Re: [PATCH 4/5] uprobes: Introduce copy_to_page()
In reply to Oleg NesterovQuoted 5 lines · Oleg Nesterov
Extract the kmap_atomic/memcpy/kunmap_atomic code from
xol_get_insn_slot()into the new simple helper,copy_to_page(). It will have more users soon.Signed-off-by Oleg Nesterov <[email protected]>
Quoted 47 lines
--- kernel/events/uprobes.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-)kernel/events/uprobes.c +8
-6@@ -180,6 +180,13 @@ static void copy_from_page(struct page *page, unsigned long vaddr, void *dst, inkunmap_atomic (kaddr); } +static void copy_to_page (struct page *page,unsigned long vaddr,const void *src,int len) +{ +void *kaddr =kmap_atomic (page); +memcpy (kaddr + (vaddr & ~PAGE_MASK ), src, len); +kunmap_atomic (kaddr); +} +static int verify_opcode (struct page *page,unsigned long vaddr,uprobe_opcode_t *new_opcode) {uprobe_opcode_t old_opcode;@@ -1204,9 +1211,7 @@ static unsigned long xol_take_insn_slot(struct xol_area *area)static unsigned long xol_get_insn_slot (struct uprobe *uprobe) {struct xol_area *area; -unsigned long offset;unsigned long xol_vaddr; -void *vaddr; area =get_xol_area ();if (!area)@@ -1217,10 +1222,7 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)return 0 ;/* Initialize the slot */ - offset = xol_vaddr & ~PAGE_MASK ; - vaddr =kmap_atomic (area->page); -memcpy (vaddr + offset, uprobe->arch.insn,MAX_UINSN_BYTES ); -kunmap_atomic (vaddr); +copy_to_page (area->page, xol_vaddr, uprobe->arch.insn,MAX_UINSN_BYTES );/* * We probably need flush_icache_user_range() but it needs vma. * This should work on supported architectures too. Signature · 1 line
1.5.5.1
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Oleg Nesterov
maintainerPATCH 5/5 uprobes: Change write_opcode() to use copy_*page()
In reply to Oleg NesterovChange write_opcode() to use copy_highpage() + copy_to_page() and simplify the code.
--- kernel/events/uprobes.c | 12 ++---------- 1 files changed, 2 insertions(+), 10 deletions(-)
kernel/events/uprobes.c +2 -10
@@ -232,7 +232,6 @@ static int write_opcode(struct mm_struct *mm, unsigned long vaddr, uprobe_opcode_t opcode) {struct page *old_page, *new_page; -void *vaddr_old, *vaddr_new;struct vm_area_struct *vma;int ret;
@@ -253,15 +252,8 @@ retry:__SetPageUptodate (new_page); -/* copy the page now that we've got it stable */ - vaddr_old =kmap_atomic (old_page); - vaddr_new =kmap_atomic (new_page); - -memcpy (vaddr_new, vaddr_old,PAGE_SIZE ); -memcpy (vaddr_new + (vaddr & ~PAGE_MASK ), &opcode,UPROBE_SWBP_INSN_SIZE ); - -kunmap_atomic (vaddr_new); -kunmap_atomic (vaddr_old); +copy_highpage (new_page, old_page); +copy_to_page (new_page, vaddr, &opcode,UPROBE_SWBP_INSN_SIZE ); ret =anon_vma_prepare (vma);if (ret)
Signature · 1 line
1.5.5.1
Re: [PATCH 5/5] uprobes: Change write_opcode() to use copy_*page()
Quoted 41 lines · Oleg Nesterov
Change
write_opcode()to usecopy_highpage()+copy_to_page()and simplify the code.Signed-off-by Oleg Nesterov <[email protected]>--- kernel/events/uprobes.c | 12 ++---------- 1 files changed, 2 insertions(+), 10 deletions(-)kernel/events/uprobes.c +2
-10@@ -232,7 +232,6 @@ static int write_opcode(struct mm_struct *mm, unsigned long vaddr, uprobe_opcode_t opcode) {struct page *old_page, *new_page; -void *vaddr_old, *vaddr_new;struct vm_area_struct *vma;int ret;@@ -253,15 +252,8 @@ retry:__SetPageUptodate (new_page); -/* copy the page now that we've got it stable */ - vaddr_old =kmap_atomic (old_page); - vaddr_new =kmap_atomic (new_page); - -memcpy (vaddr_new, vaddr_old,PAGE_SIZE ); -memcpy (vaddr_new + (vaddr & ~PAGE_MASK ), &opcode,UPROBE_SWBP_INSN_SIZE ); - -kunmap_atomic (vaddr_new); -kunmap_atomic (vaddr_old); +copy_highpage (new_page, old_page); +copy_to_page (new_page, vaddr, &opcode,UPROBE_SWBP_INSN_SIZE ); ret =anon_vma_prepare (vma);if (ret)Signature · 1 line
1.5.5.1
Re: [PATCH 5/5] uprobes: Change write_opcode() to use copy_*page()
In reply to Oleg NesterovQuoted 4 lines · Oleg Nesterov
Change
write_opcode()to usecopy_highpage()+copy_to_page()and simplify the code.Signed-off-by Oleg Nesterov <[email protected]>
Quoted 37 lines
--- kernel/events/uprobes.c | 12 ++---------- 1 files changed, 2 insertions(+), 10 deletions(-)kernel/events/uprobes.c +2
-10@@ -232,7 +232,6 @@ static int write_opcode(struct mm_struct *mm, unsigned long vaddr, uprobe_opcode_t opcode) {struct page *old_page, *new_page; -void *vaddr_old, *vaddr_new;struct vm_area_struct *vma;int ret;@@ -253,15 +252,8 @@ retry:__SetPageUptodate (new_page); -/* copy the page now that we've got it stable */ - vaddr_old =kmap_atomic (old_page); - vaddr_new =kmap_atomic (new_page); - -memcpy (vaddr_new, vaddr_old,PAGE_SIZE ); -memcpy (vaddr_new + (vaddr & ~PAGE_MASK ), &opcode,UPROBE_SWBP_INSN_SIZE ); - -kunmap_atomic (vaddr_new); -kunmap_atomic (vaddr_old); +copy_highpage (new_page, old_page); +copy_to_page (new_page, vaddr, &opcode,UPROBE_SWBP_INSN_SIZE ); ret =anon_vma_prepare (vma);if (ret)Signature · 1 line
1.5.5.1
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Re: [PATCH 0/5] kmap cleanups for uretprobes (Was: extract fill_page() and trampoline implementation)
In reply to Oleg NesterovQuoted 15 lines · Oleg Nesterov
Quoted 6 lines · Oleg Nesterov
Quoted 2 lines · Anton Arapov
+static void
fill_page(struct page *page, unsigned long offset, uprobe_opcode_t *insn)^^^^^^^^^^^^^^^Well, it should be "u8 *insn" or "char *".
Or void*.
And we have another reason for this helper.
And I really think that we need to cleanup the kmap mess in
uprobe.cbefore we add the new abuser.
Quoted 7 lines
How about this simple series?
Quoted 2 lines
sizeof(insn) ==
UPROBE_SWBP_INSN_SIZE!=MAX_UINSN_BYTES. See also the comments above.I think that
copy_to_page()added by 4/5 is what you need, and it solves the problems with typeof/sizeof. Plus it have another caller.
That's what I can't come up with. My fill_page() was indeed ugly.
Thanks!
Anton.
Quoted 2 lines
Oleg.
Oleg Nesterov
maintainerRe: [PATCH 2/7] uretprobes: extract fill_page() and trampoline implementation
In reply to Oleg NesterovQuoted 7 lines · Oleg Nesterov
Quoted 5 lines · Anton Arapov
kernel/events/uprobes.c +0
-0@@ -1139,6 +1155,10 @@ static struct xol_area *get_xol_area(void)if (!area->page) goto free_bitmap;
Quoted 6 lines
kernel/events/uprobes.c static struct xol_area *get_xol_area(void) · line 1160
Quoted 3 lines
kernel/events/uprobes.c +3
-0+/* pre-allocate for return probes */ +set_bit (0 , area->bitmap); +fill_page (area->page,0 , &insn);sizeof(insn) ==
UPROBE_SWBP_INSN_SIZE!=MAX_UINSN_BYTES. See also the comments above.
Sorry, forgot to mention... you also need to update ->slot_count.
Oleg.
PATCH 3/7 uretprobes/x86: hijack return address
In reply to Anton Arapov- Hijack the return address and replace it with a trampoline address.
RFCv5 changes:
- change the fail return code, because orig_ret_vaddr=0 is possible
- style fixup
RFCv2 changes:
- remove ->doomed flag, kill task immediately
--- arch/x86/include/asm/uprobes.h | 1 + arch/x86/kernel/uprobes.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+)
arch/x86/include/asm/uprobes.h +1 -0
@@ -55,4 +55,5 @@ extern int arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);extern bool arch_uprobe_xol_was_trapped (struct task_struct *tsk);extern int arch_uprobe_exception_notify (struct notifier_block *self,unsigned long val,void *data);extern void arch_uprobe_abort_xol (struct arch_uprobe *aup,struct pt_regs *regs); +extern unsigned long arch_uretprobe_hijack_return_addr (unsigned long trampoline_vaddr,struct pt_regs *regs);#endif /* _ASM_UPROBES_H */
arch/x86/kernel/uprobes.c +29 -0
@@ -697,3 +697,32 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)send_sig (SIGTRAP , current,0 );return ret; } + +unsigned long +arch_uretprobe_hijack_return_addr (unsigned long trampoline_vaddr,struct pt_regs *regs) +{ +int rasize, ncopied; +unsigned long orig_ret_vaddr =0 ;/* clear high bits for 32-bit apps */ + + rasize =is_ia32_task () ?4 :8 ; + ncopied =copy_from_user (&orig_ret_vaddr, (void __user *)regs->sp, rasize); +if (unlikely (ncopied)) +return -1 ; + +/* check whether address has been already hijacked */ +if (orig_ret_vaddr == trampoline_vaddr) +return orig_ret_vaddr; + + ncopied =copy_to_user ((void __user *)regs->sp, &trampoline_vaddr, rasize); +if (unlikely (ncopied)) { +if (ncopied != rasize) { +printk (KERN_ERR "uprobe: return address clobbered: " +"pid=%d, %%sp=%#lx, %%ip=%#lx\n" , + current->pid, regs->sp, regs->ip); +/* kill task immediately */ +send_sig (SIGSEGV , current,0 ); + } + } + +return orig_ret_vaddr; +}
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerRe: [PATCH 3/7] uretprobes/x86: hijack return address
arch/x86/kernel/uprobes.c bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs) · line 707
Quoted 7 lines · Anton Arapov
arch/x86/kernel/uprobes.c +6
-0+arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) +{ +int rasize, ncopied; +unsigned long orig_ret_vaddr =0 ;/* clear high bits for 32-bit apps */ + + rasize =is_ia32_task () ?4 :8 ;
Hmm.. I guess you need is_compat_task() here.
arch/x86/kernel/uprobes.c bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs) · line 719
Quoted 12 lines
arch/x86/kernel/uprobes.c +12
-0+ ncopied =copy_from_user (&orig_ret_vaddr, (void __user *)regs->sp, rasize); +if (unlikely (ncopied)) +return -1 ; + +/* check whether address has been already hijacked */ +if (orig_ret_vaddr == trampoline_vaddr) +return orig_ret_vaddr; + + ncopied =copy_to_user ((void __user *)regs->sp, &trampoline_vaddr, rasize); +if (unlikely (ncopied)) { +if (ncopied != rasize) { + printk(KERN_ERR "uprobe: return address clobbered: "
Cosmetic, but we have pr_err().
arch/x86/kernel/uprobes.c bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs) · line 723
Quoted 4 lines
arch/x86/kernel/uprobes.c +4
-0+"pid=%d, %%sp=%#lx, %%ip=%#lx\n" , + current->pid, regs->sp, regs->ip); +/* kill task immediately */ +send_sig (SIGSEGV , current,0 );
probably force_sig_info() makes more sense, but this is minor
Review
Quoted 1 line
(quoted diff) +1
-0+ }
You need to retun -1 here.
Cosmetic again, but since this function should be updated anyway, I'd suggest
ncopied = copy_to_user(...);
if (likely(!ncopied))
return orig_ret_vaddr;
if (ncopied != rasize) {
...
}
return -1;Oleg.
PATCH 4/7 uretprobes: return probe entry, prepare_uretprobe()
In reply to Anton Arapov When a uprobe with return probe consumer is hit, prepare_uretprobe()
function is invoked. It creates return_instance, hijacks return address and replaces it with the trampoline.
- Return instances are kept as stack per uprobed task.
- Return instance is dirty, when the original return address is trampoline's page vaddr (e.g. recursive call of the probed function).
N.B. it might be a good idea to introduce get_uprobe() to reflectput_uprobe() later, but it is not a subject of this patchset.
RFCv6 changes:
- rework prepare_uretprobe() logic in order to make further unwinding in handler_uretprobe() simplier.
- introduce the 'dirty' field.
RFCv5 changes:
- switch from hlist to simply linked list for tracking ->*return_uprobes.
- revert (*) from v4.
- preallocate first slot xol_area for return probes, see xol_get_area() changes.
- add get_trampoline_vaddr() helper, to emphasize area->vaddr overload.
RFCv4 changes:
- get rid of area->rp_trampoline_vaddr as it always the same as ->vaddr.
- cleanup ->return_uprobes list in uprobe_free_utask(), because the task can exit from inside the ret-probe'd function(s).
- in find_active_uprobe(): Once we inserted "int3" we must ensure that handle_swbp() will be called even if this uprobe goes away. We have the reference but it only protects uprobe itself, it can't protect agains delete_uprobe().
IOW, we must ensure that uprobe_pre_sstep_notifier() can't return 0.
RFCv3 changes:
- protected uprobe with refcounter. See atomic_inc in prepare_uretprobe() and put_uprobe() in a following patch in handle_uretprobe().
RFCv2 changes:
- get rid of ->return_consumers member from struct uprobe, introduce rp_handler() in consumer.
--- include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 91 insertions(+), 1 deletion(-)
include/linux/uprobes.h +1 -0
@@ -69,6 +69,7 @@ struct uprobe_task {enum uprobe_task_state state;struct arch_uprobe_task autask; +struct return_instance *return_instances;struct uprobe *active_uprobe;unsigned long xol_vaddr;
kernel/events/uprobes.c +90 -1
134 lines not on this page · raw message
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerRe: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
I'll try to read this series later. Just one note...
Quoted 2 lines · Anton Arapov
IOW, we must ensure that
uprobe_pre_sstep_notifier()can't return 0.
Yes, but I do not see this change?
Quoted 17 lines
kernel/events/uprobes.c +17
-0+static void prepare_uretprobe (struct uprobe *uprobe,struct pt_regs *regs) +{ +struct return_instance *ri; +struct uprobe_task *utask; +struct xol_area *area; +unsigned long trampoline_vaddr; +unsigned long prev_ret_vaddr, ret_vaddr; + + area =get_xol_area (); +if (!area) +return ; + + utask =get_utask (); +if (!utask) +return ; + + prev_ret_vaddr =-1 ;
kernel/events/uprobes.c static unsigned long get_trampoline_vaddr(struct xol_area *area) · line 1405
Quoted 10 lines
kernel/events/uprobes.c +10
-0+if (utask->return_instances) + prev_ret_vaddr = utask->return_instances->orig_ret_vaddr; + + ri =kzalloc (sizeof (struct return_instance ),GFP_KERNEL ); +if (!ri) +return ; + + ri->dirty = false; + trampoline_vaddr =get_trampoline_vaddr (area); + ret_vaddr =arch_uretprobe_hijack_return_addr (trampoline_vaddr, regs);
OK, but you need to ensure that this code can be compiled on poweprc.
Oleg.
Re: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
Quoted 29 lines · Oleg Nesterov
I'll try to read this series later. Just one note...
Quoted 2 lines · Anton Arapov
IOW, we must ensure that
uprobe_pre_sstep_notifier()can't return 0.Yes, but I do not see this change?
Quoted 21 lines
kernel/events/uprobes.c +21
-0+static void prepare_uretprobe (struct uprobe *uprobe,struct pt_regs *regs) +{ +struct return_instance *ri; +struct uprobe_task *utask; +struct xol_area *area; +unsigned long trampoline_vaddr; +unsigned long prev_ret_vaddr, ret_vaddr; + + area =get_xol_area (); +if (!area) +return ; + + utask =get_utask (); +if (!utask) +return ; + + prev_ret_vaddr =-1 ; +if (utask->return_instances) + prev_ret_vaddr = utask->return_instances->orig_ret_vaddr; + + ri =kzalloc (sizeof (struct return_instance ),GFP_KERNEL );
Quoted 8 lines
kernel/events/uprobes.c static unsigned long get_trampoline_vaddr(struct xol_area *area) · line 1405
Quoted 6 lines
kernel/events/uprobes.c +6
-0+if (!ri) +return ; + + ri->dirty = false; + trampoline_vaddr =get_trampoline_vaddr (area); + ret_vaddr =arch_uretprobe_hijack_return_addr (trampoline_vaddr, regs);OK, but you need to ensure that this code can be compiled on poweprc.
It does compile, unlike the obvious arch_uretprobe_hijack_return_addr() where I'll look for the Ananth's help, perhaps.
Anton.
Oleg Nesterov
maintainerRe: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
Quoted 10 lines · Anton Arapov
Quoted 5 lines · Oleg Nesterov
kernel/events/uprobes.c static unsigned long get_trampoline_vaddr(struct xol_area *area) · line 1405
Quoted 3 lines
kernel/events/uprobes.c +3
-0+ ri->dirty = false; + trampoline_vaddr =get_trampoline_vaddr (area); + ret_vaddr =arch_uretprobe_hijack_return_addr (trampoline_vaddr, regs);OK, but you need to ensure that this code can be compiled on poweprc.
It does compile, unlike the obvious
arch_uretprobe_hijack_return_addr()where I'll look for the Ananth's help, perhaps.
Just make the default weak function which returns -1.
After that Ananth can send the (hopefully simple) patch with the powerpc implementation.
Oleg.
Oleg Nesterov
maintainerRe: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
In reply to Anton ArapovQuoted 7 lines · Anton Arapov
kernel/events/uprobes.c +1
-0@@ -1513,20 +1596,26 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) {struct uprobe_consumer *uc;int remove =UPROBE_HANDLER_REMOVE ; +bool prep = false;/* prepare return uprobe, when needed */ int rc =0 ;
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1609
Quoted 10 lines
kernel/events/uprobes.c +3
-1down_read (&uprobe->register_rwsem);for (uc = uprobe->consumers; uc; uc = uc->next) {if (uc->handler) rc = uc->handler (uc, regs); -else +else { + prep |= true; remove =0 ; + }
This looks wrong (see also my reply to 1/7).
What if uc->handler != NULL and uc->ret_handler != NULL ? We need
if (uc->rer_handler) need_prep = true;
Oleg.
Oleg Nesterov
maintainerRe: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
In reply to Anton Arapovkernel/events/uprobes.c void uprobe_free_utask(struct task_struct *t) · line 1337
Quoted 13 lines · Anton Arapov
void
uprobe_free_utask(struct task_struct *t)
{kernel/events/uprobes.c +2
-0struct uprobe_task *utask = t->utask; +struct return_instance *ri, *tmp;if (!utask)return ;@@ -1325,6 +1334,15 @@ void uprobe_free_utask(struct task_struct *t)if (utask->active_uprobe)put_uprobe (utask->active_uprobe); + ri = utask->return_instances;
You also need to nullify ->return_instances before return, otherwise it can be use-after-freed later.
uprobe_free_utask() can also be called when the task execs.
kernel/events/uprobes.c void uprobe_free_utask(struct task_struct *t) · line 1344
Quoted 7 lines
kernel/events/uprobes.c +7
-0+while (ri) { +put_uprobe (ri->uprobe); + + tmp = ri; + ri = ri->next; +kfree (tmp); + }
This is really minor, but I can't resist. Both put_uprobe() and kfree() work with the same object, it would be more clean to use the same var. Say,
while (ri) {
tmp = ri;
ri = ri->next;
put_uprobe(tmp->uprobe);
kfree(tmp);
}Quoted 2 lines
kernel/events/uprobes.c +2
-0+static void prepare_uretprobe (struct uprobe *uprobe,struct pt_regs *regs) +{
...
Quoted 39 lines
kernel/events/uprobes.c +39
-0+ + prev_ret_vaddr =-1 ; +if (utask->return_instances) + prev_ret_vaddr = utask->return_instances->orig_ret_vaddr; + + ri =kzalloc (sizeof (struct return_instance ),GFP_KERNEL ); +if (!ri) +return ; + + ri->dirty = false; + trampoline_vaddr =get_trampoline_vaddr (area); + ret_vaddr =arch_uretprobe_hijack_return_addr (trampoline_vaddr, regs); + +/* +* We don't want to keep trampoline address in stack, rather keep the +* original return address of first caller thru all the consequent +* instances. This also makes breakpoint unwrapping easier. +*/ +if (ret_vaddr == trampoline_vaddr) { +if (likely (prev_ret_vaddr !=-1 )) { + ri->dirty = true; + ret_vaddr = prev_ret_vaddr; + }else { +/* +* This situation is not possible. Likely we have an +* attack from user-space. Die. +*/ +printk (KERN_ERR "uprobe: something went wrong " +"pid/tgid=%d/%d" , current->pid, current->tgid); +send_sig (SIGSEGV , current,0 ); +kfree (ri); +return ; + } + } + +if (likely (ret_vaddr !=-1 )) { +atomic_inc (&uprobe->ref); + ri->uprobe = uprobe; + ri->orig_ret_vaddr = ret_vaddr;
kernel/events/uprobes.c static unsigned long get_trampoline_vaddr(struct xol_area *area) · line 1442
Quoted 10 lines
kernel/events/uprobes.c +10
-0+ +/* add instance to the stack */ + ri->next = utask->return_instances; + utask->return_instances = ri; + +return ; + } + +kfree (ri); +}
Anton, this really doesn't look clear/clean. Why do you need prev_ret_vaddr in advance? Why do you need it at all? why do you delay the "ret_vaddr == -1" errorcheck?
And ->dirty looks confusing... perhaps ->chained ?
ri = kzalloc(...);
if (!ri)
return;
ret_vaddr = arch_uretprobe_hijack_return_addr(...);
if (ret_vaddr == -1)
goto err;
if (ret_vaddr == trampoline_vaddr) {
if (!utask->return_instances) {
// This situation is not possible.
// (not sure we should send SIGSEGV)
pr_warn(...);
goto err;
}
ri->chained = true;
ret_vaddr = utask->return_instances->orig_ret_vaddr;
}
fill-ri-and-add-push-it;
return;
err:
kfree(ri);
return;Oleg.
Re: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
Quoted 117 lines · Oleg Nesterov
kernel/events/uprobes.c void uprobe_free_utask(struct task_struct *t) · line 1337
Quoted 13 lines · Anton Arapov
void
uprobe_free_utask(struct task_struct *t)
{kernel/events/uprobes.c +2
-0struct uprobe_task *utask = t->utask; +struct return_instance *ri, *tmp;if (!utask)return ;@@ -1325,6 +1334,15 @@ void uprobe_free_utask(struct task_struct *t)if (utask->active_uprobe)put_uprobe (utask->active_uprobe); + ri = utask->return_instances;You also need to nullify ->return_instances before return, otherwise it can be use-after-freed later.
uprobe_free_utask()can also be called when the task execs.kernel/events/uprobes.c void uprobe_free_utask(struct task_struct *t) · line 1344
Quoted 7 lines
kernel/events/uprobes.c +7
-0+while (ri) { +put_uprobe (ri->uprobe); + + tmp = ri; + ri = ri->next; +kfree (tmp); + }This is really minor, but I can't resist. Both
put_uprobe()andkfree()work with the same object, it would be more clean to use the same var. Say,while (ri) { tmp = ri; ri = ri->next; put_uprobe(tmp->uprobe); kfree(tmp); }Quoted 2 lines
kernel/events/uprobes.c +2
-0+static void prepare_uretprobe (struct uprobe *uprobe,struct pt_regs *regs) +{...
Quoted 39 lines
kernel/events/uprobes.c +39
-0+ + prev_ret_vaddr =-1 ; +if (utask->return_instances) + prev_ret_vaddr = utask->return_instances->orig_ret_vaddr; + + ri =kzalloc (sizeof (struct return_instance ),GFP_KERNEL ); +if (!ri) +return ; + + ri->dirty = false; + trampoline_vaddr =get_trampoline_vaddr (area); + ret_vaddr =arch_uretprobe_hijack_return_addr (trampoline_vaddr, regs); + +/* +* We don't want to keep trampoline address in stack, rather keep the +* original return address of first caller thru all the consequent +* instances. This also makes breakpoint unwrapping easier. +*/ +if (ret_vaddr == trampoline_vaddr) { +if (likely (prev_ret_vaddr !=-1 )) { + ri->dirty = true; + ret_vaddr = prev_ret_vaddr; + }else { +/* +* This situation is not possible. Likely we have an +* attack from user-space. Die. +*/ +printk (KERN_ERR "uprobe: something went wrong " +"pid/tgid=%d/%d" , current->pid, current->tgid); +send_sig (SIGSEGV , current,0 ); +kfree (ri); +return ; + } + } + +if (likely (ret_vaddr !=-1 )) { +atomic_inc (&uprobe->ref); + ri->uprobe = uprobe; + ri->orig_ret_vaddr = ret_vaddr;kernel/events/uprobes.c static unsigned long get_trampoline_vaddr(struct xol_area *area) · line 1442
Quoted 10 lines
kernel/events/uprobes.c +10
-0+ +/* add instance to the stack */ + ri->next = utask->return_instances; + utask->return_instances = ri; + +return ; + } + +kfree (ri); +}Anton, this really doesn't look clear/clean. Why do you need
prev_ret_vaddrin advance? Why do you need it at all? why do you delay the "ret_vaddr== -1" errorcheck?And ->dirty looks confusing... perhaps ->chained ?
ri = kzalloc(...); if (!ri) return; ret_vaddr = arch_uretprobe_hijack_return_addr(...); if (ret_vaddr == -1) goto err; if (ret_vaddr == trampoline_vaddr) { if (!utask->return_instances) { // This situation is not possible. // (not sure we should send SIGSEGV) pr_warn(...); goto err; } ri->chained = true; ret_vaddr = utask->return_instances->orig_ret_vaddr;
Quoted 8 lines
} fill-ri-and-add-push-it; return; err: kfree(ri); return;
I will do the appropriate changes. Thanks!
Anton.
Quoted 2 lines
Oleg.
Re: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
In reply to Oleg NesterovQuoted 1 line · Oleg Nesterov
On 03/22, Anton Arapov wrote:
[snip]
Quoted 17 lines
And ->dirty looks confusing... perhaps ->chained ?
ri = kzalloc(...); if (!ri) return; ret_vaddr = arch_uretprobe_hijack_return_addr(...); if (ret_vaddr == -1) goto err; if (ret_vaddr == trampoline_vaddr) { if (!utask->return_instances) { // This situation is not possible. // (not sure we should send SIGSEGV) pr_warn(...); goto err; }
If we don't send SIGSEGV, does it make sense to restore the original
return address that was just hijacked? So that we just decline setting the breakpoint for this very case.
Anton.
Quoted 14 lines
ri->chained = true; ret_vaddr = utask->return_instances->orig_ret_vaddr; } fill-ri-and-add-push-it; return; err: kfree(ri); return;Oleg.
Re: [PATCH 4/7] uretprobes: return probe entry, prepare_uretprobe()
Quoted 21 lines · Anton Arapov
Quoted 1 line · Oleg Nesterov
On 03/22, Anton Arapov wrote:
[snip]
Quoted 17 lines
And ->dirty looks confusing... perhaps ->chained ?
ri = kzalloc(...); if (!ri) return; ret_vaddr = arch_uretprobe_hijack_return_addr(...); if (ret_vaddr == -1) goto err; if (ret_vaddr == trampoline_vaddr) { if (!utask->return_instances) { // This situation is not possible. // (not sure we should send SIGSEGV) pr_warn(...); goto err; }
Quoted 3 lines
If we don't send SIGSEGV, does it make sense to restore the original
return address that was just hijacked? So that we just decline setting the breakpoint for this very case.
disregard this one. we have no address to restore at that moment. :)
Quoted 16 lines
Anton.
Quoted 14 lines
ri->chained = true; ret_vaddr = utask->return_instances->orig_ret_vaddr; } fill-ri-and-add-push-it; return; err: kfree(ri); return;Oleg.
PATCH 5/7 uretprobes: return probe exit, invoke handlers
In reply to Anton Arapov Uretprobe handlers are invoked when the trampoline is hit, on completion the
trampoline is replaced with the saved return address and the uretprobe instance deleted.
RFCv6 changes:
- rework handle_uretprobe()
RFCv5 changes:
- switch to simply linked list ->return_uprobes
- rework handle_uretprobe()
RFCv4 changes:
- check, whether utask is not NULL in handle_uretprobe()
- get rid of area->rp_trampoline_vaddr
- minor handle_uretprobe() fixups
RFCv3 changes:
- protected uprobe with refcounter. See put_uprobe() in handle_uretprobe() that reflects increment in prepare_uretprobe()
RFCv2 changes:
- get rid of ->return_consumers member from struct uprobe, introduce rp_handler() in consumer instead
--- kernel/events/uprobes.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-)
kernel/events/uprobes.c +59 -1
@@ -1623,6 +1623,56 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs)up_read (&uprobe->register_rwsem); } +static void handler_uretprobe_chain (struct uprobe *uprobe,struct pt_regs *regs) +{ +struct uprobe_consumer *uc; + +down_read (&uprobe->register_rwsem); +for (uc = uprobe->consumers; uc; uc = uc->next) { +if (uc->rp_handler) + uc->rp_handler (uc, regs); + } +up_read (&uprobe->register_rwsem); +} + +static void handle_uretprobe (struct xol_area *area,struct pt_regs *regs) +{ +struct uprobe_task *utask; +struct return_instance *ri, *tmp; +unsigned long prev_ret_vaddr; + + utask =get_utask (); +if (!utask) +return ; + + ri = utask->return_instances; +if (!ri) +return ; + +instruction_pointer_set (regs, ri->orig_ret_vaddr); + +while (ri) { +if (ri->uprobe->consumers) +handler_uretprobe_chain (ri->uprobe, regs); + +put_uprobe (ri->uprobe); + tmp = ri; + prev_ret_vaddr = tmp->orig_ret_vaddr; + ri = ri->next; +kfree (tmp); + +if (!ri || ri->dirty == false) { +/* +* This is the first return uprobe (chronologically) +* pushed for this particular instance of the probed +* function. +*/ + utask->return_instances = ri; +return ; + } + } +} +/* * Run handler and ask thread to singlestep. * Ensure all non-fatal signals cannot interrupt thread while it singlesteps.
@@ -1631,11 +1681,19 @@ static void handle_swbp(struct pt_regs *regs) {struct uprobe *uprobe;unsigned long bp_vaddr; +struct xol_area *area;int uninitialized_var (is_swbp ); bp_vaddr =uprobe_get_swbp_addr (regs); - uprobe =find_active_uprobe (bp_vaddr, &is_swbp); + area =get_xol_area (); +if (area) { +if (bp_vaddr ==get_trampoline_vaddr (area)) { +handle_uretprobe (area, regs); +return ; + } + } + uprobe =find_active_uprobe (bp_vaddr, &is_swbp);if (!uprobe) {if (is_swbp >0 ) {/* No matching uprobe; signal SIGTRAP. */
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerRe: [PATCH 5/7] uretprobes: return probe exit, invoke handlers
Quoted 4 lines · Anton Arapov
kernel/events/uprobes.c +3
-0+static void handle_uretprobe (struct xol_area *area,struct pt_regs *regs) +{ +struct uprobe_task *utask;
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1650
Quoted 10 lines
kernel/events/uprobes.c +10
-0+struct return_instance *ri, *tmp; +unsigned long prev_ret_vaddr; + + utask =get_utask (); +if (!utask) +return ; + + ri = utask->return_instances; +if (!ri) +return ;
Hmm. I am wondering what should the caller (handle_swbp) do in this case...
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1656
Quoted 6 lines
kernel/events/uprobes.c +6
-0+ +instruction_pointer_set (regs, ri->orig_ret_vaddr); + +while (ri) { +if (ri->uprobe->consumers) +handler_uretprobe_chain (ri->uprobe, regs);
I'd suggest to either remove this check or move it into handler_uretprobe_chain().
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1660
Quoted 4 lines
kernel/events/uprobes.c +4
-0+ +put_uprobe (ri->uprobe); + tmp = ri; + prev_ret_vaddr = tmp->orig_ret_vaddr;
For what? It seems that prev_ret_vaddr should be simply killed.
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1662
Quoted 2 lines
kernel/events/uprobes.c +2
-0+ ri = ri->next; +kfree (tmp);
Another case when you do put_uprobe/kfree using the different vars... Once again, the code is correct but imho a bit confusing.
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1672
Quoted 9 lines
kernel/events/uprobes.c +9
-0+if (!ri || ri->dirty == false) { +/* +* This is the first return uprobe (chronologically) +* pushed for this particular instance of the probed +* function. +*/ + utask->return_instances = ri; +return ; + }
Else? we simply return without updating ->return_instances which points to the freed element(s) ? OK, this must not be possible but this is not obvious...
And the fact you check "ri != NULL" twice doesn't look very nice. We already checked ri != NULL before while(ri), we have to do this anyway for instruction_pointer_set(). Perhaps do/whild or even for (;;) + break would be more clean. But this is minor.
I am not sure the logic is correct. OK. suppose that ->return_instances = NULL.
The task hits the rp breakoint. After that
return_instances -> { .dirty = false }The task hits the same breakoint before return (tail call), now we have
return_instances -> { .dirty = true } -> { .dirty = false }Then it returns and handle_uretprobe() should unwind the whole stack. But, it seems, the main loop will stop after the 1st iteration?
Ignoring the fact you need put_uprobe/kfree, it seems that we should do something like this,
do {
handler_uretprobe_chain(...);
if (!ri->dirty) // not chained
break;
ri = ri->next;
} while (ri);
utask->return_instances = ri;No?
kernel/events/uprobes.c static void handle_swbp(struct pt_regs *regs) · line 1688
Quoted 10 lines
kernel/events/uprobes.c +2
-1@@ -1631,11 +1681,19 @@ static void handle_swbp(struct pt_regs *regs) {struct uprobe *uprobe;unsigned long bp_vaddr; +struct xol_area *area;int uninitialized_var (is_swbp ); bp_vaddr =uprobe_get_swbp_addr (regs); - uprobe =find_active_uprobe (bp_vaddr, &is_swbp); + area =get_xol_area ();
Why?
No, we do not want this heavy and potentially unnecessary get_xol_area(),
kernel/events/uprobes.c static void handle_swbp(struct pt_regs *regs) · line 1689
Quoted 1 line
kernel/events/uprobes.c +1
-0+if (area) {
Just check uprobes_state.xol_area != NULL instead. If it is NULL we simply should not call handle_uretprobe().
Or perhaps get_trampoline_vaddr() should simply return -1 if ->xol_area == NULL.
kernel/events/uprobes.c static void handle_swbp(struct pt_regs *regs) · line 1690
Quoted 1 line
kernel/events/uprobes.c +1
-0+if (bp_vaddr ==get_trampoline_vaddr (area)) {
I just noticed get_trampoline_vaddr() takes an argument... It should not, I think.
Oleg.
Oleg Nesterov
maintainerRe: [PATCH 5/7] uretprobes: return probe exit, invoke handlers
The last comment, I promise ;)
Quoted 19 lines · Oleg Nesterov
Quoted 4 lines · Anton Arapov
kernel/events/uprobes.c +3
-0+static void handle_uretprobe (struct xol_area *area,struct pt_regs *regs) +{ +struct uprobe_task *utask;kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1650
Quoted 10 lines
kernel/events/uprobes.c +10
-0+struct return_instance *ri, *tmp; +unsigned long prev_ret_vaddr; + + utask =get_utask (); +if (!utask) +return ; + + ri = utask->return_instances; +if (!ri) +return ;Hmm. I am wondering what should the caller (
handle_swbp) do in this case...
And you do not actually need get_utask(), just check current->utask.
handle_uretprobe() must not be called if either ->utask or ->return_instances is NULL. This can only happen if we have a bug, or user-space tries to fool the kernel.
Perhaps it makes sense to add pr_warn().
Oleg.
Re: [PATCH 5/7] uretprobes: return probe exit, invoke handlers
In reply to Oleg NesterovQuoted 17 lines · Oleg Nesterov
Quoted 4 lines · Anton Arapov
kernel/events/uprobes.c +3
-0+static void handle_uretprobe (struct xol_area *area,struct pt_regs *regs) +{ +struct uprobe_task *utask;kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1650
Quoted 10 lines
kernel/events/uprobes.c +10
-0+struct return_instance *ri, *tmp; +unsigned long prev_ret_vaddr; + + utask =get_utask (); +if (!utask) +return ; + + ri = utask->return_instances; +if (!ri) +return ;Hmm. I am wondering what should the caller (
handle_swbp) do in this case...
Not sure as well... Will look into it.
Quoted 9 lines
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1656
Quoted 6 lines
kernel/events/uprobes.c +6
-0+ +instruction_pointer_set (regs, ri->orig_ret_vaddr); + +while (ri) { +if (ri->uprobe->consumers) +handler_uretprobe_chain (ri->uprobe, regs);I'd suggest to either remove this check or move it into
handler_uretprobe_chain().
Quoted 5 lines
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1660
Quoted 4 lines
kernel/events/uprobes.c +4
-0+ +put_uprobe (ri->uprobe); + tmp = ri; + prev_ret_vaddr = tmp->orig_ret_vaddr;For what? It seems that
prev_ret_vaddrshould be simply killed.
Both above are leftovers I've overlooked before git-send... :(
Quoted 4 lines
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1662
Quoted 2 lines
kernel/events/uprobes.c +2
-0+ ri = ri->next; +kfree (tmp);Another case when you do put_uprobe/kfree using the different vars... Once again, the code is correct but imho a bit confusing.
I agree will change it and align with the code in uprobe_free_utask()
Quoted 48 lines
kernel/events/uprobes.c static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs) · line 1672
Quoted 9 lines
kernel/events/uprobes.c +9
-0+if (!ri || ri->dirty == false) { +/* +* This is the first return uprobe (chronologically) +* pushed for this particular instance of the probed +* function. +*/ + utask->return_instances = ri; +return ; + }Else? we simply return without updating ->return_instances which points to the freed element(s) ? OK, this must not be possible but this is not obvious...
And the fact you check "ri != NULL" twice doesn't look very nice. We already checked ri != NULL before while(ri), we have to do this anyway for
instruction_pointer_set(). Perhaps do/whild or even for (;;) + break would be more clean. But this is minor.I am not sure the logic is correct. OK. suppose that ->return_instances = NULL.
The task hits the rp breakoint. After that
return_instances -> { .dirty = false }The task hits the same breakoint before return (tail call), now we have
return_instances -> { .dirty = true } -> { .dirty = false }Then it returns and
handle_uretprobe()should unwind the whole stack. But, it seems, the main loop will stop after the 1st iteration?Ignoring the fact you need put_uprobe/kfree, it seems that we should do something like this,
do { handler_uretprobe_chain(...); if (!ri->dirty) // not chained break; ri = ri->next; } while (ri); utask->return_instances = ri;
Quoted 1 line
No?
Oleg, Do you mean
do {
handler_uretprobe_chain(...);
ri = ri->next;
if (!ri->dirty) // not chained
break;
} while (ri);
utask->return_instances = ri;
otherwise we stuck with the first instance in stack. ...and perhaps for(;;) would be 'more beautiful' here?
Quoted 15 lines
kernel/events/uprobes.c static void handle_swbp(struct pt_regs *regs) · line 1688
Quoted 11 lines
kernel/events/uprobes.c +2
-1@@ -1631,11 +1681,19 @@ static void handle_swbp(struct pt_regs *regs) {struct uprobe *uprobe;unsigned long bp_vaddr; +struct xol_area *area;int uninitialized_var (is_swbp ); bp_vaddr =uprobe_get_swbp_addr (regs); - uprobe =find_active_uprobe (bp_vaddr, &is_swbp); + area =get_xol_area ();Why?
No, we do not want this heavy and potentially unnecessaryget_xol_area(),
Quoted 7 lines
kernel/events/uprobes.c static void handle_swbp(struct pt_regs *regs) · line 1689
Quoted 1 line
kernel/events/uprobes.c +1
-0+if (area) {Just check uprobes_state.xol_area != NULL instead. If it is NULL we simply should not call
handle_uretprobe().Or perhaps
get_trampoline_vaddr()should simply return -1 if ->xol_area == NULL.
right.
Quoted 6 lines
kernel/events/uprobes.c static void handle_swbp(struct pt_regs *regs) · line 1690
Quoted 2 lines
kernel/events/uprobes.c +1
-0+if (bp_vaddr ==get_trampoline_vaddr (area)) {I just noticed
get_trampoline_vaddr()takes an argument... It should not, I think.
Yes, at this place we must have *area allocated. And I agree with
your arguments, I will remove *area argument from get_trampoline_vaddr() and handle_uretprobe() it makes sense to me as well.
Anton
Quoted 2 lines
Oleg.
Oleg Nesterov
maintainerRe: [PATCH 5/7] uretprobes: return probe exit, invoke handlers
Quoted 31 lines · Anton Arapov
Quoted 14 lines · Oleg Nesterov
Ignoring the fact you need put_uprobe/kfree, it seems that we should do something like this,
do { handler_uretprobe_chain(...); if (!ri->dirty) // not chained break; ri = ri->next; } while (ri); utask->return_instances = ri;Quoted 1 line
No?
Oleg, Do you mean
(quoted diff) +0
-0do { handler_uretprobe_chain(...); ri = ri->next; if (!ri->dirty) // not chained break; } while (ri); utask->return_instances = ri;
Quoted 1 line
otherwise we stuck with the first instance in stack.
Not sure I understand... but it is very possible I missed something.
But the pseudo code I wrote is not correct, I meant
utask->return_instances = ri->next;
after the main loop.
Quoted 1 line
...and perhaps for(;;) would be 'more beautiful' here?
Oh, I would not mind either way. In fact we do not really need ri != NULL check inside the loop (again, unless I am confused). We must see a non-chained entry in the stack unless we have a bug.
Oleg.
Re: [PATCH 5/7] uretprobes: return probe exit, invoke handlers
Quoted 34 lines · Oleg Nesterov
Quoted 31 lines · Anton Arapov
Quoted 14 lines · Oleg Nesterov
Ignoring the fact you need put_uprobe/kfree, it seems that we should do something like this,
do { handler_uretprobe_chain(...); if (!ri->dirty) // not chained break; ri = ri->next; } while (ri); utask->return_instances = ri;Quoted 1 line
No?
Oleg, Do you mean
(quoted diff) +0
-0do { handler_uretprobe_chain(...); ri = ri->next; if (!ri->dirty) // not chained break; } while (ri); utask->return_instances = ri;Quoted 1 line
otherwise we stuck with the first instance in stack.
Quoted 7 lines
Not sure I understand... but it is very possible I missed something.
But the pseudo code I wrote is not correct, I meant
utask->return_instances = ri->next;after the main loop.
This all makes sense now. Thanks.
Anton.
PATCH 6/7 uretprobes: limit the depth of return probe nestedness
In reply to Anton Arapov Unlike the kretprobes we can't trust userspace, thus must have
protection from user space attacks, this patch limits the return probes nestedness as a simple remedy for it.
The intention is to have KISS and bare minimum solution for the
initial implementation in order to not complicate the uretprobes code.
In the future we may come up with more sophisticated solution that
should remove this depth limitation, however it is not easy task and lays beyond this patchset. It should consider things like: breakpoint address lays outside the stack and stack growth direction, longjmp, sigaltstack... be able to clean up return instances.
--- include/linux/uprobes.h | 3 +++ kernel/events/uprobes.c | 11 +++++++++++ 2 files changed, 14 insertions(+)
include/linux/uprobes.h +3 -0
@@ -38,6 +38,8 @@ struct inode;#define UPROBE_HANDLER_REMOVE 1#define UPROBE_HANDLER_MASK 1 +#define MAX_URETPROBE_DEPTH 64 +enum uprobe_filter_ctx {UPROBE_FILTER_REGISTER ,UPROBE_FILTER_UNREGISTER ,
@@ -70,6 +72,7 @@ struct uprobe_task {struct arch_uprobe_task autask;struct return_instance *return_instances; +unsigned int depth;struct uprobe *active_uprobe;unsigned long xol_vaddr;
kernel/events/uprobes.c +11 -0
@@ -1392,6 +1392,13 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)if (!utask)return ; +if (utask->depth >=MAX_URETPROBE_DEPTH ) { +printk_ratelimited (KERN_INFO "urpobe: reached the return probe" +" depth limit pid/tgid=%d/%d\n" , current->pid, + current->tgid); +return ; + } + prev_ret_vaddr =-1 ;if (utask->return_instances) prev_ret_vaddr = utask->return_instances->orig_ret_vaddr;
@@ -1431,6 +1438,8 @@ static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs) ri->uprobe = uprobe; ri->orig_ret_vaddr = ret_vaddr; + utask->depth++; +/* add instance to the stack */ ri->next = utask->return_instances; utask->return_instances = ri;
@@ -1661,6 +1670,8 @@ static void handle_uretprobe(struct xol_area *area, struct pt_regs *regs) ri = ri->next;kfree (tmp); + utask->depth--; +if (!ri || ri->dirty == false) {/* * This is the first return uprobe (chronologically)
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerRe: [PATCH 6/7] uretprobes: limit the depth of return probe nestedness
Quoted 3 lines · Anton Arapov
Unlike the kretprobes we can't trust userspace, thus must have
protection from user space attacks,
Plus the user-space have the "unlimited" stack, it looks simply impossible to handle the deep recursion correctly.
Lets consider the simplest case,
void probed_func(int x)
{
if (--x)
probed_func(x);
}We could add ret_instance->recursion_counter to handle this case and avoid kmalloc(). But this way we won't be able to implement the new features like data-pouch.
Quoted 11 lines
this patch limits the return
probes nestedness as a simple remedy for it.
The intention is to have KISS and bare minimum solution for the
initial implementation in order to not complicate the uretprobes code.In the future we may come up with more sophisticated solution that
should remove this depth limitation, however it is not easy task and lays beyond this patchset. It should consider things like: breakpoint address lays outside the stack and stack growth direction, longjmp, sigaltstack... be able to clean up return instances.
I agree. Perhaps we should at least try to do something more clever in future, but this needs another (certainly nontrivial) series, we should not mix this with the initial implementation, it should be as simple as possible.
Oleg.
PATCH 7/7 uretprobes: implemented, thus remove -ENOSYS
In reply to Anton Arapovenclose return probes implementation
v3 changes: (the patch is introduced in v3)
- remove 'TODO' as return probes implemented now
--- kernel/events/uprobes.c | 4 ---- 1 file changed, 4 deletions(-)
kernel/events/uprobes.c +0 -4
@@ -840,10 +840,6 @@ int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *if (!uc->handler && !uc->rp_handler)return -EINVAL ; -/* TODO: Implement return probes */ -if (uc->rp_handler) -return -ENOSYS ; -/* Racy, just to catch the obvious mistakes */ if (offset >i_size_read (inode))return -EINVAL ;
Signature · 1 line
1.8.1.4
Re: [PATCH 7/7] uretprobes: implemented, thus remove -ENOSYS
please disregard this one, not sure how it managed to hide within the rest ones...
sry,
Anton.
Quoted 28 lines · Anton Arapov
enclose return probes implementation
v3 changes: (the patch is introduced in v3)
- remove 'TODO' as return probes implemented nowSigned-off-by Anton Arapov <[email protected]>--- kernel/events/uprobes.c | 4 ---- 1 file changed, 4 deletions(-)kernel/events/uprobes.c +0
-4@@ -840,10 +840,6 @@ int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *if (!uc->handler && !uc->rp_handler)return -EINVAL ; -/* TODO: Implement return probes */ -if (uc->rp_handler) -return -ENOSYS ; -/* Racy, just to catch the obvious mistakes */ if (offset >i_size_read (inode))return -EINVAL ;Signature · 1 line
1.8.1.4
PATCH 7/7 uretprobes: remove -ENOSYS as return probes implemented
In reply to Anton ArapovEnclose return probes implementation.
--- kernel/events/uprobes.c | 4 ---- 1 file changed, 4 deletions(-)
kernel/events/uprobes.c +0 -4
@@ -840,10 +840,6 @@ int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *if (!uc->handler && !uc->rp_handler)return -EINVAL ; -/* TODO: Implement return probes */ -if (uc->rp_handler) -return -ENOSYS ; -/* Racy, just to catch the obvious mistakes */ if (offset >i_size_read (inode))return -EINVAL ;
Signature · 1 line
1.8.1.4
Oleg Nesterov
maintainerQuoted 3 lines · Anton Arapov
this is core implementation of the uretprobes. This enables a breakpoint
on function's return for the tools such as perf.
And, before we ask Ingo to pull, we should teach trace_uprobe.c to handle "perf -x func%return".
This looks simple, but probably we need to add the additional "ulong bp_vaddr" argument for rp_handler().
Oleg.
Quoted 2 lines · Oleg Nesterov
This looks simple, but probably we need to add the additional "ulong
bp_vaddr" argument forrp_handler().
FWIW, in SystemTap we don't currently do anything that would need bp_vaddr. The uprobe_consumer already gives the context, so I'm not sure what else you would do with the *entry* address of the function that just returned. Perhaps you could emulate an additional top line on a stacktrace, but it doesn't tell you where in the function you were.
Josh
Quoted 9 lines · Josh Stone
Quoted 2 lines · Oleg Nesterov
This looks simple, but probably we need to add the additional "ulong
bp_vaddr" argument forrp_handler().FWIW, in SystemTap we don't currently do anything that would need
bp_vaddr. Theuprobe_consumeralready gives the context, so I'm not sure what else you would do with the *entry* address of the function that just returned. Perhaps you could emulate an additional top line on a stacktrace, but it doesn't tell you where in the function you were.
IIUC, Oleg have stressed that perf and trace events requires
additional code to leverage the return probes. IOW this patchset implements core only and to use it we need to teach the perf about rp_handler().
And I don't see why we would need additional parameter forrp_handler() as well.
Anton
Oleg Nesterov
maintainerQuoted 5 lines · Anton Arapov
IIUC, Oleg have stressed that perf and trace events requires
additional code to leverage the return probes. IOW this patchset implements core only and to use it we need to teach the perf aboutrp_handler().
Yes,
Quoted 2 lines
And I don't see why we would need additional parameter for
rp_handler()as well.
trace_uprobe should use the same fmt = "(%lx <- %lx)" to teport "entry <- exit" info, like trace_kprobe does.
And note that, unlike kprobe, the same function can be mmapped at different address by different (or even the same) tasks.
But mostly I just wanted to say that yes, we will also teach trace_uprobe to use the new feature before we ask to merge this code.
Oleg.
Oleg Nesterov
maintainerQuoted 7 lines · Josh Stone
Quoted 2 lines · Oleg Nesterov
This looks simple, but probably we need to add the additional "ulong
bp_vaddr" argument forrp_handler().FWIW, in SystemTap we don't currently do anything that would need
bp_vaddr.
I know. This is not for systemtap.
Quoted 3 lines
The
uprobe_consumeralready gives the context, so I'm not sure what else you would do with the *entry* address of the function that just returned.
context - yes, but not address (by the time ret_handler is called).
Oleg.