uprobes: pre-filtering
Ingo, please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/oleg/misc uprobes/core
Mostly pre-filtering. This needs more work and perhaps more functionality. In particular, perhaps dup_mmap() should remove the unwanted breakpoints. And we can add more ->filter() hooks to, say, speedup uprobe_register(). Plus we can do some optimizations to avoid register_for_each_vma() in case when we know that all mm's were previously acked/nacked.
Srikar, the only patch you did not ack explicitely is 1fecb96d "Do not allocate current->utask unnecessary", but afaics you do not object.
And the patch from Josh which exports uprobe_register/unregister for modules. Christoph (cc'ed) doesn't like this change, but I disagree. Whatever you think about systemtap it is the widely used tool, and uprobes can have other out-of-tree users. This is like kprobes, kprobe_register() is exported but it doesn't have a modular in-kernel user too. I do not see why should we limit the usage of uprobes.
Josh Stone (1):
uprobes: Add exports for module use
Oleg Nesterov (26):
uprobes: Move __set_bit(UPROBE_SKIP_SSTEP) into alloc_uprobe()
uprobes: Kill the "uprobe != NULL" check in uprobe_unregister()
uprobes: Kill the pointless inode/uc checks in register/unregister
uprobes: Kill uprobe_consumer->filter()
uprobes: Introduce filter_chain()
uprobes: _unregister() should always do register_for_each_vma(false)
uprobes: _register() should always do register_for_each_vma(true)
uprobes: Introduce uprobe->register_rwsem
uprobes: Change filter_chain() to iterate ->consumers list
uprobes: Kill UPROBE_RUN_HANDLER flag
uprobes: Kill uprobe->copy_mutex
uprobes: Kill uprobe_events, use RB_EMPTY_ROOT() instead
uprobes: Introduce uprobe_is_active()
uprobes: Kill uprobes_mutex[], separate alloc_uprobe() and __uprobe_register()
uprobes: Rationalize the usage of filter_chain()
uprobes: Reintroduce uprobe_consumer->filter()
uprobes: Teach handler_chain() to filter out the probed task uprobes/x86: Change __skip_sstep() to actually skip the whole insn
uprobes: Change handle_swbp() to expose bp_vaddr to handler_chain()
uprobes: Move alloc_page() from xol_add_vma() to xol_alloc_area()
uprobes: Fold xol_alloc_area() into get_xol_area()
uprobes: Turn add_utask() into get_utask()
uprobes: Do not play with utask in xol_get_insn_slot()
uprobes: Fix utask->xol_vaddr leak in pre_ssout()
uprobes: Do not allocate current->utask unnecessary
uprobes: Kill the bogus IS_ERR_VALUE(xol_vaddr) check
arch/x86/kernel/uprobes.c | 4 +- include/linux/uprobes.h | 17 ++- kernel/events/uprobes.c | 433 ++++++++++++++++++++++--------------------- kernel/ptrace.c | 6 + kernel/trace/trace_uprobe.c | 5 +- 5 files changed, 243 insertions(+), 222 deletions(-)
Quoted 54 lines · Oleg Nesterov
Ingo, please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/oleg/miscuprobes/coreMostly pre-filtering. This needs more work and perhaps more functionality. In particular, perhaps
dup_mmap()should remove the unwanted breakpoints. And we can add more ->filter() hooks to, say, speedupuprobe_register(). Plus we can do some optimizations to avoidregister_for_each_vma()in case when we know that all mm's were previously acked/nacked.Srikar, the only patch you did not ack explicitely is
1fecb96d"Do not allocatecurrent->utaskunnecessary", but afaics you do not object.And the patch from Josh which exports uprobe_register/unregister for modules. Christoph (cc'ed) doesn't like this change, but I disagree. Whatever you think about systemtap it is the widely used tool, and uprobes can have other out-of-tree users. This is like kprobes,
kprobe_register()is exported but it doesn't have a modular in-kernel user too. I do not see why should we limit the usage of uprobes.Josh Stone (1):
uprobes: Add exports for module useOleg Nesterov (26):
uprobes: Move__set_bit(UPROBE_SKIP_SSTEP)intoalloc_uprobe()
uprobes: Kill the "uprobe != NULL" check inuprobe_unregister()
uprobes: Kill the pointless inode/uc checks in register/unregister
uprobes: Killuprobe_consumer->filter()
uprobes: Introducefilter_chain()
uprobes:_unregister()should always doregister_for_each_vma(false)
uprobes:_register()should always doregister_for_each_vma(true)
uprobes: Introduceuprobe->register_rwsem
uprobes: Changefilter_chain()to iterate ->consumers list
uprobes: KillUPROBE_RUN_HANDLERflag
uprobes: Killuprobe->copy_mutex
uprobes: Killuprobe_events, useRB_EMPTY_ROOT()instead
uprobes: Introduceuprobe_is_active()
uprobes: Kill uprobes_mutex[], separatealloc_uprobe()and__uprobe_register()
uprobes: Rationalize the usage offilter_chain()
uprobes: Reintroduceuprobe_consumer->filter()
uprobes: Teachhandler_chain()to filter out the probed task uprobes/x86: Change__skip_sstep()to actually skip the whole insn
uprobes: Changehandle_swbp()to exposebp_vaddrtohandler_chain()
uprobes: Movealloc_page()fromxol_add_vma()toxol_alloc_area()
uprobes: Foldxol_alloc_area()intoget_xol_area()
uprobes: Turnadd_utask()intoget_utask()
uprobes: Do not play with utask inxol_get_insn_slot()
uprobes: Fixutask->xol_vaddrleak inpre_ssout()
uprobes: Do not allocatecurrent->utaskunnecessary
uprobes: Kill the bogusIS_ERR_VALUE(xol_vaddr)check
Quoted 6 lines
arch/x86/kernel/uprobes.c | 4 +- include/linux/uprobes.h | 17 ++- kernel/events/uprobes.c | 433 ++++++++++++++++++++++--------------------- kernel/ptrace.c | 6 + kernel/trace/trace_uprobe.c | 5 +- 5 files changed, 243 insertions(+), 222 deletions(-)
The kernel side looks good to me - but how does 'perf uprobe' make use of it in practice, how can I test it?
Thanks,
Ingo
Hello Ingo,
Quoted 64 lines · Ingo Molnar
Quoted 54 lines · Oleg Nesterov
Ingo, please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/oleg/miscuprobes/coreMostly pre-filtering. This needs more work and perhaps more functionality. In particular, perhaps
dup_mmap()should remove the unwanted breakpoints. And we can add more ->filter() hooks to, say, speedupuprobe_register(). Plus we can do some optimizations to avoidregister_for_each_vma()in case when we know that all mm's were previously acked/nacked.Srikar, the only patch you did not ack explicitely is
1fecb96d"Do not allocatecurrent->utaskunnecessary", but afaics you do not object.And the patch from Josh which exports uprobe_register/unregister for modules. Christoph (cc'ed) doesn't like this change, but I disagree. Whatever you think about systemtap it is the widely used tool, and uprobes can have other out-of-tree users. This is like kprobes,
kprobe_register()is exported but it doesn't have a modular in-kernel user too. I do not see why should we limit the usage of uprobes.Josh Stone (1):
uprobes: Add exports for module useOleg Nesterov (26):
uprobes: Move__set_bit(UPROBE_SKIP_SSTEP)intoalloc_uprobe()
uprobes: Kill the "uprobe != NULL" check inuprobe_unregister()
uprobes: Kill the pointless inode/uc checks in register/unregister
uprobes: Killuprobe_consumer->filter()
uprobes: Introducefilter_chain()
uprobes:_unregister()should always doregister_for_each_vma(false)
uprobes:_register()should always doregister_for_each_vma(true)
uprobes: Introduceuprobe->register_rwsem
uprobes: Changefilter_chain()to iterate ->consumers list
uprobes: KillUPROBE_RUN_HANDLERflag
uprobes: Killuprobe->copy_mutex
uprobes: Killuprobe_events, useRB_EMPTY_ROOT()instead
uprobes: Introduceuprobe_is_active()
uprobes: Kill uprobes_mutex[], separatealloc_uprobe()and__uprobe_register()
uprobes: Rationalize the usage offilter_chain()
uprobes: Reintroduceuprobe_consumer->filter()
uprobes: Teachhandler_chain()to filter out the probed task uprobes/x86: Change__skip_sstep()to actually skip the whole insn
uprobes: Changehandle_swbp()to exposebp_vaddrtohandler_chain()
uprobes: Movealloc_page()fromxol_add_vma()toxol_alloc_area()
uprobes: Foldxol_alloc_area()intoget_xol_area()
uprobes: Turnadd_utask()intoget_utask()
uprobes: Do not play with utask inxol_get_insn_slot()
uprobes: Fixutask->xol_vaddrleak inpre_ssout()
uprobes: Do not allocatecurrent->utaskunnecessary
uprobes: Kill the bogusIS_ERR_VALUE(xol_vaddr)checkQuoted 6 lines
arch/x86/kernel/uprobes.c | 4 +- include/linux/uprobes.h | 17 ++- kernel/events/uprobes.c | 433 ++++++++++++++++++++++--------------------- kernel/ptrace.c | 6 + kernel/trace/trace_uprobe.c | 5 +- 5 files changed, 243 insertions(+), 222 deletions(-)
Quoted 2 lines
The kernel side looks good to me - but how does 'perf uprobe' make use of it in practice, how can I test it?
I'm not sure whether you looking into testing specific changes in this pull, but in general, syntax is:
perf probe -x /lib64/libc.so.6 malloc
perf record -e probe_libc:p_malloc -aR sleep 30
hope this is what you was looking for,
Anton.
Quoted 67 lines · Anton Arapov
Hello Ingo,
Quoted 64 lines · Ingo Molnar
Quoted 60 lines · Oleg Nesterov
Ingo, please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/oleg/miscuprobes/coreMostly pre-filtering. This needs more work and perhaps more functionality. In particular, perhaps
dup_mmap()should remove the unwanted breakpoints. And we can add more ->filter() hooks to, say, speedupuprobe_register(). Plus we can do some optimizations to avoidregister_for_each_vma()in case when we know that all mm's were previously acked/nacked.Srikar, the only patch you did not ack explicitely is
1fecb96d"Do not allocatecurrent->utaskunnecessary", but afaics you do not object.And the patch from Josh which exports uprobe_register/unregister for modules. Christoph (cc'ed) doesn't like this change, but I disagree. Whatever you think about systemtap it is the widely used tool, and uprobes can have other out-of-tree users. This is like kprobes,
kprobe_register()is exported but it doesn't have a modular in-kernel user too. I do not see why should we limit the usage of uprobes.Josh Stone (1):
uprobes: Add exports for module useOleg Nesterov (26):
uprobes: Move__set_bit(UPROBE_SKIP_SSTEP)intoalloc_uprobe()
uprobes: Kill the "uprobe != NULL" check inuprobe_unregister()
uprobes: Kill the pointless inode/uc checks in register/unregister
uprobes: Killuprobe_consumer->filter()
uprobes: Introducefilter_chain()
uprobes:_unregister()should always doregister_for_each_vma(false)
uprobes:_register()should always doregister_for_each_vma(true)
uprobes: Introduceuprobe->register_rwsem
uprobes: Changefilter_chain()to iterate ->consumers list
uprobes: KillUPROBE_RUN_HANDLERflag
uprobes: Killuprobe->copy_mutex
uprobes: Killuprobe_events, useRB_EMPTY_ROOT()instead
uprobes: Introduceuprobe_is_active()
uprobes: Kill uprobes_mutex[], separatealloc_uprobe()and__uprobe_register()
uprobes: Rationalize the usage offilter_chain()
uprobes: Reintroduceuprobe_consumer->filter()
uprobes: Teachhandler_chain()to filter out the probed task uprobes/x86: Change__skip_sstep()to actually skip the whole insn
uprobes: Changehandle_swbp()to exposebp_vaddrtohandler_chain()
uprobes: Movealloc_page()fromxol_add_vma()toxol_alloc_area()
uprobes: Foldxol_alloc_area()intoget_xol_area()
uprobes: Turnadd_utask()intoget_utask()
uprobes: Do not play with utask inxol_get_insn_slot()
uprobes: Fixutask->xol_vaddrleak inpre_ssout()
uprobes: Do not allocatecurrent->utaskunnecessary
uprobes: Kill the bogusIS_ERR_VALUE(xol_vaddr)checkarch/x86/kernel/uprobes.c | 4 +- include/linux/uprobes.h | 17 ++- kernel/events/uprobes.c | 433 ++++++++++++++++++++++--------------------- kernel/ptrace.c | 6 + kernel/trace/trace_uprobe.c | 5 +- 5 files changed, 243 insertions(+), 222 deletions(-)
Quoted 5 lines
Quoted 2 lines
The kernel side looks good to me - but how does 'perf uprobe' make use of it in practice, how can I test it?
I'm not sure whether you looking into testing specific changes in this pull, [...]
Yes, I was curious about specifically testing the filtering callback changes in this pull request.
Thanks,
Ingo
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Oleg Nesterov
Ingo, please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/oleg/miscuprobes/core
Quoted 8 lines
Quoted 5 lines
Mostly pre-filtering. This needs more work and perhaps more functionality. In particular, perhaps
dup_mmap()should remove the unwanted breakpoints. And we can add more ->filter() hooks to, say, speedupuprobe_register(). Plus we can do some optimizations to avoidregister_for_each_vma()in case when we know that all mm's were previously acked/nacked.The kernel side looks good to me - but how does 'perf uprobe' make use of it in practice, how can I test it?
Unfortunately, currently there is no in-kernel user of pre-filtering.
I'll try to implement the pid-base filtering at least for tracing/uprobe_events, but this needs a time. Not only I am not familiar with this code, I am not sure how this interface should actually look. And I agree, perf should be able to use it somehow, perhaps at least to allow to probe a single task/mm.
Oleg.
Quoted 20 lines · Oleg Nesterov
Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Oleg Nesterov
Ingo, please pull from
git://git.kernel.org/pub/scm/linux/kernel/git/oleg/miscuprobes/coreQuoted 8 lines
Quoted 5 lines
Mostly pre-filtering. This needs more work and perhaps more functionality. In particular, perhaps
dup_mmap()should remove the unwanted breakpoints. And we can add more ->filter() hooks to, say, speedupuprobe_register(). Plus we can do some optimizations to avoidregister_for_each_vma()in case when we know that all mm's were previously acked/nacked.The kernel side looks good to me - but how does 'perf uprobe' make use of it in practice, how can I test it?
Unfortunately, currently there is no in-kernel user of pre-filtering.
Quoted 6 lines
I'll try to implement the pid-base filtering at least for tracing/uprobe_events, but this needs a time. Not only I am not familiar with this code, I am not sure how this interface should actually look. And I agree, perf should be able to use it somehow, perhaps at least to allow to probe a single task/mm.
Would be nice to get something minimal/simple going, so that it can be tested, etc.
Thanks,
Ingo
Quoted 13 lines · Ingo Molnar
Quoted 9 lines · Oleg Nesterov
Unfortunately, currently there is no in-kernel user of pre-filtering.
I'll try to implement the pid-base filtering at least for tracing/uprobe_events, but this needs a time. Not only I am not familiar with this code, I am not sure how this interface should actually look. And I agree, perf should be able to use it somehow, perhaps at least to allow to probe a single task/mm.
Quoted 2 lines
Would be nice to get something minimal/simple going, so that it can be tested, etc.
Heh, I understand.
I do not see anything simple to implement... I'll try to think. Srikar, do you have any idea?
All I can say right now: I'll send you the patches when I have them ;) Can't promise this will be soon.
_Perhaps_, as a first step, we can simply change create_trace_uprobe() so that it would be possible to specify list-of-pids at creation time... Not sure this actually makes sense.
Oleg.
Quoted 22 lines · Oleg Nesterov
Quoted 13 lines · Ingo Molnar
Quoted 9 lines · Oleg Nesterov
Unfortunately, currently there is no in-kernel user of pre-filtering.
I'll try to implement the pid-base filtering at least for tracing/uprobe_events, but this needs a time. Not only I am not familiar with this code, I am not sure how this interface should actually look. And I agree, perf should be able to use it somehow, perhaps at least to allow to probe a single task/mm.
Quoted 2 lines
Would be nice to get something minimal/simple going, so that it can be tested, etc.
Heh, I understand.
I do not see anything simple to implement... I'll try to think. Srikar, do you have any idea?
Quoted 7 lines
All I can say right now: I'll send you the patches when I have them ;) Can't promise this will be soon.
_Perhaps_, as a first step, we can simply change
create_trace_uprobe()so that it would be possible to specify list-of-pids at creation time... Not sure this actually makes sense.
[ Adding Masami and Steven since they have other ideas / inputs ]
I can see two not-so-easy alternatives
Reuse the event's post filtering meta data and convert it to a prefiltering handler. Convertion looks very tricky
probe_event_enable() seems to be the right place to do this. tu->call.filter would have the post filtering data utc->filter needs to be set to the pre-filtering handler.
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
Quoted 2 lines
Oleg.
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Quoted 3 lines · Srikar Dronamraju
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
That's the obvious path to go - why add something to the kernel if user-space cannot make use of it?
Thanks,
Ingo
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Srikar Dronamraju
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
That's the obvious path to go - why add something to the kernel if user-space cannot make use of it?
This is what I am going to (try to) do, but I am not sure if this makes sense...
For the start, can't we teach 'uprobe_events' file to accept, say,
'p file:0x1234 pid=1 other-opts'
for the start? This looks simple enough, and I after looked into tools/perf it seems that perf can be changed too.
What do you think?
Then we can extend 'pid=' option to accept the list of pids, perhaps.
In the long term we probably need uprobes/pid_filter or something like this, it should allow to add/del pid dynamically. I really do not know.
Oleg.
Quoted 14 lines · Oleg Nesterov
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Srikar Dronamraju
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
That's the obvious path to go - why add something to the kernel if user-space cannot make use of it?
This is what I am going to (try to) do, but I am not sure if this makes sense...
Quoted 8 lines
For the start, can't we teach '
uprobe_events' file to accept, say,'p file:0x1234 pid=1 other-opts'for the start? This looks simple enough, and I after looked into
tools/perfit seems that perf can be changed too.What do you think?
Sounds sensible and functional to me.
Quoted 6 lines
Then we can extend 'pid=' option to accept the list of pids, perhaps.
In the long term we probably need uprobes/pid_filter or something like this, it should allow to add/del pid dynamically. I really do not know.
For now removing+adding a new one should be enough to 'change' a uprobe, right?
Thanks,
Ingo
Quoted 19 lines · Ingo Molnar
Quoted 16 lines · Oleg Nesterov
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Srikar Dronamraju
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
That's the obvious path to go - why add something to the kernel if user-space cannot make use of it?
This is what I am going to (try to) do, but I am not sure if this makes sense...
For the start, can't we teach '
uprobe_events' file to accept, say,
Quoted 8 lines
Quoted 6 lines
'p file:0x1234 pid=1 other-opts'for the start? This looks simple enough, and I after looked into
tools/perfit seems that perf can be changed too.What do you think?
Sounds sensible and functional to me.
Great, thanks.
Quoted 9 lines
Quoted 6 lines
Then we can extend 'pid=' option to accept the list of pids, perhaps.
In the long term we probably need uprobes/pid_filter or something like this, it should allow to add/del pid dynamically. I really do not know.
For now removing+adding a new one should be enough to 'change' a uprobe, right?
Yes, yes. I meant that obviously we can do more/better to filter-out the tasks we do not want to probe. But this need more changes, and more importantly this needs more discussion about API/ABI/etc.
Thanks.
Oleg.
Quoted 11 lines · Oleg Nesterov
Quoted 9 lines · Ingo Molnar
Quoted 3 lines · Srikar Dronamraju
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
That's the obvious path to go - why add something to the kernel if user-space cannot make use of it?
Quoted 6 lines
This is what I am going to (try to) do, but I am not sure if this makes sense...
For the start, can't we teach '
uprobe_events' file to accept, say,'p file:0x1234 pid=1 other-opts'
I think this would be a very good start
The only downside, I see is we would have add and remove to change the filter. Something like perf record will not be able to dynamically change the filter parameter.
Quoted 10 lines
for the start? This looks simple enough, and I after looked into
tools/perfit seems that perf can be changed too.What do you think?
Then we can extend 'pid=' option to accept the list of pids, perhaps.
In the long term we probably need uprobes/pid_filter or something like this, it should allow to add/del pid dynamically. I really do not know.
Yes, I think having a file like uprobes/pid_filter would be able to dynamically change the filter. Probably when we code this up should we make this something more generic otherwise, we might end up with uid_filter, sid_filter ..
Steven,
Are you still pursuing multiple ftrace buffers that you proposed at LPC 2012?
In which case, this new filter should also be per buffer.
Signature · 2 lines
Thanks and Regards Srikar Dronamraju
Quoted 6 lines · Srikar Dronamraju
Quoted 3 lines · Oleg Nesterov
This is what I am going to (try to) do, but I am not sure if this makes sense...
Quoted 6 lines
Quoted 3 lines
For the start, can't we teach '
uprobe_events' file to accept, say,'p file:0x1234 pid=1 other-opts'I think this would be a very good start
Good.
So will you agree with 3/4 and 3/4 ?
Quoted 3 lines
The only downside, I see is we would have add and remove to change the filter. Something like perf record will not be able to dynamically change the filter parameter.
Yes, this is only the first step.
Oleg.
Quoted 3 lines · Srikar Dronamraju
The other alternative is to extend the current abi and pass the prefilter option. Should we extend the abi for userspace tracing is obviously debatable.
Hmm, afaik, ftrace has set_ftrace_pid and perf record also has --pid option. I think you can hook those to set pre-filter option at first. Of course, since those are universal options, we may need other per-event pid filter.
Thank you,
Signature · 4 lines
Masami HIRAMATSU IT Management Research Dept. Linux Technology Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected]
Quoted 5 lines · Oleg Nesterov
I'll try to implement the pid-base filtering at least for tracing/uprobe_events, but this needs a time. Not only I am not familiar with this code, I am not sure how this interface should actually look. And I agree, perf should be able to use it somehow, perhaps at least to allow to probe a single task/mm.
Even without changing perf's interface, it already has constraints for its child processes (versus --all-cpus) or for specific pid/tid/uid. Pre-filtering could help enforce and optimize those constraints.
Quoted 11 lines · Josh Stone
Quoted 5 lines · Oleg Nesterov
I'll try to implement the pid-base filtering at least for tracing/uprobe_events, but this needs a time. Not only I am not familiar with this code, I am not sure how this interface should actually look. And I agree, perf should be able to use it somehow, perhaps at least to allow to probe a single task/mm.
Even without changing perf's interface, it already has constraints for its child processes (versus --all-cpus) or for specific pid/tid/uid. Pre-filtering could help enforce and optimize those constraints.
This is what I meant. The question is how can I do this.
Oleg.