perf: prctl(PR_TASK_PERF_EVENTS_DISABLE) has no effect
as there was no proposed solution that helped me in response to the same post by Andrew Steets, I would like to know if it is possible to disable/enable perf event counters from the source code?
calling prctl(PR_TASK_PERF_EVENTS_DISABLE) has no effect, nor does compiling with -fno-omit-frame-pointer option.
It would be extremely useful to disable perf event counters for some parts of code and re-enable them for other parts of code, like:
prctl(PR_TASK_PERF_EVENTS_DISABLE);
// not important for performance analysis codeprctl(PR_TASK_PERF_EVENTS_ENABLE);
// code that needs to be analysed
and then, run perf:
$ perf record ./program $ perf report
Can anyone tell me how can I enable such functionality? Thank you
--Oleg
Quoted 15 lines · Iegorov Oleg
as there was no proposed solution that helped me in response to the same post by Andrew Steets, I would like to know if it is possible to disable/enable perf event counters from the source code?
calling
prctl(PR_TASK_PERF_EVENTS_DISABLE)has no effect, nor does compiling with -fno-omit-frame-pointer option.It would be extremely useful to disable perf event counters for some parts of code and re-enable them for other parts of code, like:
prctl(PR_TASK_PERF_EVENTS_DISABLE);
// not important for performance analysis codeprctl(PR_TASK_PERF_EVENTS_ENABLE);
// code that needs to be analysed
Quoted 6 lines
and then, run perf:
$ perf record ./program $ perf reportCan anyone tell me how can I enable such functionality?
So, the kernel bits to do this in a pretty quirky way are there, see:
https://lkml.org/lkml/2012/1/30/99
but the librarization bits are definitely non-obvious to do and it's no surprise that it has not been done yet.
Regular 'perf record' in itself is not self-profiling - it's another task profiling you, so we cannot blanket allow PR_TASK_PERF_EVENTS_DISABLE to disable profiling.
What we might want to do instead on the kernel side to offer the functionality you are asking for is to enable 'public/weak' events be created by the profiler on an opt-in basis, which can be turned off by child tasks as well via PR_TASK_PERF_EVENTS_DISABLE.
On the profiling workflow side it would work in a very simple way, like this:
perf record --self-profiling ./my-app
In your app you stick in appropriately placed
PR_TASK_PERF_EVENTS_DISABLE/ENABLE calls.
On the technical side perf record creates events with that struct perf_event_attr::self_profiling flag set to 1. (the flag is disabled by default)
The PR_TASK_PERF_EVENTS_DISABLE code in the modified kernel then iterates through all events and disables those that have this flag set, not just the ones owned by this task.
Maybe someone on Cc: would be interested in implementing this new perf events feature?
Thanks,
Ingo
Quoted 3 lines · Ingo Molnar
Maybe someone on Cc: would be interested in implementing this new perf events feature?
Why would we go build new kernel interfaces because userspace is silly?
It really isn't that hard to make userspace do what is needed, it just takes a bit of work.
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Ingo Molnar
Maybe someone on Cc: would be interested in implementing this new perf events feature?
Why would we go build new kernel interfaces because userspace is silly?
Because it's (much) easier to use the existing perf tools almost as-is instead of librarizing your own.
It would also allow other usecases, like self-profiling a library and then profiling it within the context of a larger app that you don't want to rebuild and which dynamically links this library.
It also allows system-wide profiling after you've modified a library to self-profile, while your suggestion does not allow that.
Quoted 2 lines
It really isn't that hard to make userspace do what is needed, it just takes a bit of work.
Even if your suggested solution was available (it isn't), my suggested approach is easier to use and covers more usecases.
User-space expecting the kernel to provide usable and minimal interfaces is not 'being silly'. It's the fundamental task of a kernel to provide them.
Thanks,
Ingo
Quoted 10 lines · Ingo Molnar
Quoted 7 lines · Peter Zijlstra
Quoted 3 lines · Ingo Molnar
Maybe someone on Cc: would be interested in implementing this new perf events feature?
Why would we go build new kernel interfaces because userspace is silly?
Quoted 7 lines
Because it's (much) easier to use the existing perf tools almost as-is instead of librarizing your own.
It would also allow other usecases, like self-profiling a library and then profiling it within the context of a larger app that you don't want to rebuild and which dynamically links this library.
Uhm.. why not? For the first use proper self profiling, for the second do a regular 3rd party profile.
Quoted 3 lines
It also allows system-wide profiling after you've modified a library to self-profile, while your suggestion does not allow that.
But its no long self-profiling when some other process is involved. And system wide is definitely not self.
Quoted 9 lines
Quoted 2 lines
It really isn't that hard to make userspace do what is needed, it just takes a bit of work.
Even if your suggested solution was available (it isn't), my suggested approach is easier to use and covers more usecases.
User-space expecting the kernel to provide usable and minimal interfaces is not 'being silly'. It's the fundamental task of a kernel to provide them.
Bloating the interface for something that is already well possible is.
I really don't see the problem, other than that people simply don't want to do work.
Quoted 29 lines · Peter Zijlstra
Quoted 7 lines · Ingo Molnar
Quoted 5 lines · Peter Zijlstra
Quoted 2 lines · Ingo Molnar
Maybe someone on Cc: would be interested in implementing this new perf events feature?
Why would we go build new kernel interfaces because userspace is silly?
Quoted 7 lines
Because it's (much) easier to use the existing perf tools almost as-is instead of librarizing your own.
It would also allow other usecases, like self-profiling a library and then profiling it within the context of a larger app that you don't want to rebuild and which dynamically links this library.
Uhm.. why not? For the first use proper self profiling, for the second do a regular 3rd party profile.
Quoted 3 lines
It also allows system-wide profiling after you've modified a library to self-profile, while your suggestion does not allow that.
But its no long self-profiling when some other process is involved. And system wide is definitely not self.
Quoted 5 lines
Quoted 2 lines
It really isn't that hard to make userspace do what is needed, it just takes a bit of work.
Even if your suggested solution was available (it isn't), my suggested approach is easier to use and covers more usecases.
Quoted 7 lines
Quoted 3 lines
User-space expecting the kernel to provide usable and minimal interfaces is not 'being silly'. It's the fundamental task of a kernel to provide them.
Bloating the interface for something that is already well possible is.
I really don't see the problem, other than that people simply don't want to do work.
I totally agree with Ingo. The key words here are "much easier to use".
Moreover, in my use case, it would add a lot of complexity to rebuild the application each time a new performance event is added/removed from the event set (if I understood correctly Peter's approach).
Regards,
Oleg
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines
It also allows system-wide profiling after you've modified a library to self-profile, while your suggestion does not allow that.
But its no long self-profiling when some other process is involved. And system wide is definitely not self.
If I'm a library developer and want to self-profile my modifications, then it's entirely appropriate to do a system-wide profile to see the system-wide use of this library!
System-wide and self-profiling is not exclusive. Think of it as a narrow, precise filter applied to a given area of functionality only.
AFAICS the alternative, under your method, would be to recompile every single app in the system - that's cumbersome beyond imagination, I wouldn't even call it a solution, let alone a quality implementation.
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
It really isn't that hard to make userspace do what is needed, it just takes a bit of work.
Even if your suggested solution was available (it isn't), my suggested approach is easier to use and covers more usecases.
Quoted 6 lines
Quoted 3 lines
User-space expecting the kernel to provide usable and minimal interfaces is not 'being silly'. It's the fundamental task of a kernel to provide them.
Bloating the interface for something that is already well possible is.
There's no 'bloat' worth speaking off: a single bit out of an already allocated bitmap, plus a single check in an already existing loop, plus a single new command-line flag to the tooling side and minimal glue.
There's no runtime overhead to any other perf functionality, at all.
Thanks,
Ingo
Quoted 33 lines · Ingo Molnar
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines
It also allows system-wide profiling after you've modified a library to self-profile, while your suggestion does not allow that.
But its no long self-profiling when some other process is involved. And system wide is definitely not self.
If I'm a library developer and want to self-profile my modifications, then it's entirely appropriate to do a system-wide profile to see the system-wide use of this library!
System-wide and self-profiling is not exclusive. Think of it as a narrow, precise filter applied to a given area of functionality only.
AFAICS the alternative, under your method, would be to recompile every single app in the system - that's cumbersome beyond imagination, I wouldn't even call it a solution, let alone a quality implementation.
Quoted 11 lines
Quoted 10 lines
Quoted 2 lines
It really isn't that hard to make userspace do what is needed, it just takes a bit of work.
Even if your suggested solution was available (it isn't), my suggested approach is easier to use and covers more usecases.
User-space expecting the kernel to provide usable and minimal interfaces is not 'being silly'. It's the fundamental task of a kernel to provide them.
Quoted 7 lines
Quoted 2 lines
Bloating the interface for something that is already well possible is.
There's no 'bloat' worth speaking off: a single bit out of an already allocated bitmap, plus a single check in an already existing loop, plus a single new command-line flag to the tooling side and minimal glue.
I tried it some time ago because I had a user who was very interested in getting rid of initialization overhead in measurements.
I have a little library too to self instrument, but it's still a lot more work than adding a ioctl or two.
The patch is actually broken (causes occasional crashes) and does not implement the opt in bit (but that would be easy to add). And it's against a quite old kernel.
Also had to do the disabling only after exec to avoid some issues.
But it worked well enough for some tests.
There are likely better ways to implement this
(or least ways that don't crash :-)
Frederic's generic contexts also sound nice.
The nice thing is that combined with some LD_PRELOAD tricks you can even use it with unmodified binaries.
(not a submission or anything, just FYI)
-Andi
http://firstfloor.org/~andi/perf-disabled-1
Signature · 1 line
[email protected] -- Speaking for myself only
Quoted 6 lines · Ingo Molnar
Quoted 2 lines
Bloating the interface for something that is already well possible is.
There's no 'bloat' worth speaking off: a single bit out of an already allocated bitmap, plus a single check in an already existing loop,
Uhm, no. The existing prctl() loop is over the fd's the task owns, you want a loop over the fd's that monitor you. This needs new prctl()s at the very least.
Quoted 11 lines · Peter Zijlstra
Quoted 6 lines · Ingo Molnar
Quoted 2 lines
Bloating the interface for something that is already well possible is.
There's no 'bloat' worth speaking off: a single bit out of an already allocated bitmap, plus a single check in an already existing loop,
Uhm, no. The existing
prctl()loop is over the fd's the task owns, you want a loop over the fd's that monitor you. This needs new prctl()s at the very least.
We could add a new prctl if you think, but I thought to not complicate it and offer it as a simple extension of the semantics to loop over active events. No existing binary's behavior will change.
Thanks,
Ingo
Quoted 10 lines · Ingo Molnar
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines · Ingo Molnar
Quoted 2 lines
Bloating the interface for something that is already well possible is.
There's no 'bloat' worth speaking off: a single bit out of an already allocated bitmap, plus a single check in an already existing loop,
Quoted 8 lines
Quoted 3 lines
Uhm, no. The existing
prctl()loop is over the fd's the task owns, you want a loop over the fd's that monitor you. This needs new prctl()s at the very least.We could add a new prctl if you think, but I thought to not complicate it and offer it as a simple extension of the semantics to loop over active events. No existing binary's behavior will change.
It muddles up the semantics of the existing prctl()s though.
Ideally we'd simply remove the current ones though, I don't think anybody actually uses them and as an owner you actually have all the fds to call ioctl() on.
This would let us get rid of the entire event->owner, event->owner_entry and task->perf_event_list mess. See perf_event_exit_task() and perf_release() for why I'd love that crap to go away.
[RFD] perf: events defined contexts (was Re: perf: prctl(PR_TASK_PERF_EVENTS_DISABLE) has no effect)
In reply to Ingo MolnarQuoted 55 lines · Ingo Molnar
Quoted 15 lines · Iegorov Oleg
as there was no proposed solution that helped me in response to the same post by Andrew Steets, I would like to know if it is possible to disable/enable perf event counters from the source code?
calling
prctl(PR_TASK_PERF_EVENTS_DISABLE)has no effect, nor does compiling with -fno-omit-frame-pointer option.It would be extremely useful to disable perf event counters for some parts of code and re-enable them for other parts of code, like:
prctl(PR_TASK_PERF_EVENTS_DISABLE);
// not important for performance analysis codeprctl(PR_TASK_PERF_EVENTS_ENABLE);
// code that needs to be analysedQuoted 6 lines
and then, run perf:
$ perf record ./program $ perf reportCan anyone tell me how can I enable such functionality?
So, the kernel bits to do this in a pretty quirky way are there, see:
https://lkml.org/lkml/2012/1/30/99but the librarization bits are definitely non-obvious to do and it's no surprise that it has not been done yet.
Regular 'perf record' in itself is not self-profiling - it's another task profiling you, so we cannot blanket allow
PR_TASK_PERF_EVENTS_DISABLEto disable profiling.What we might want to do instead on the kernel side to offer the functionality you are asking for is to enable 'public/weak' events be created by the profiler on an opt-in basis, which can be turned off by child tasks as well via
PR_TASK_PERF_EVENTS_DISABLE.On the profiling workflow side it would work in a very simple way, like this:
perf record --self-profiling ./my-appIn your app you stick in appropriately placed
PR_TASK_PERF_EVENTS_DISABLE/ENABLE calls.On the technical side perf record creates events with that struct
perf_event_attr::self_profilingflag set to 1. (the flag is disabled by default)
Quoted 6 lines
The
PR_TASK_PERF_EVENTS_DISABLEcode in the modified kernel then iterates through all events and disables those that have this flag set, not just the ones owned by this task.Maybe someone on Cc: would be interested in implementing this new perf events feature?
The problem is more general than that I think.
We need to be able to define finer grained contexts than just "task" and/or "CPU".
And reusing events themselves would be a nice interface.
For example create 3 events:
A = irq entry tracepoint
B = irq exit tracepoint
C = cpu-cycles
And say: I want to count cpu-cycles when event A fires and stop counting when B fires.
With that you can count cpu cycles on irqs.
You could use any event you want to define your contexts: lock, functions, etc...
And even uprobes to define areas in userspace to profile. Would that solve the initial problem in the thread? Like hook on function library entry/exit?
I talked about that to Jiri Olsa several times. May be he would be interested in implementing this. I posted some patchets one year ago but got sidetracked.
This can give you an idea from where we can start: https://lkml.org/lkml/2011/3/14/346
Jiri, would you be interested in working on this?
Re: [RFD] perf: events defined contexts (was Re: perf: prctl(PR_TASK_PERF_EVENTS_DISABLE) has no effect)
On Fri, Jul 27, 2012 at 01:56:36PM +0200, Frederic Weisbecker wrote:
SNIP
Quoted 23 lines
The problem is more general than that I think.
We need to be able to define finer grained contexts than just "task" and/or "CPU".And reusing events themselves would be a nice interface.
For example create 3 events:
A = irq entry tracepoint
B = irq exit tracepoint
C = cpu-cyclesAnd say: I want to count cpu-cycles when event A fires and stop counting when B fires.
With that you can count cpu cycles on irqs.
You could use any event you want to define your contexts: lock, functions, etc...
And even uprobes to define areas in userspace to profile. Would that solve the initial problem in the thread? Like hook on function library entry/exit?
Quoted 6 lines
I talked about that to Jiri Olsa several times. May be he would be interested in implementing this. I posted some patchets one year ago but got sidetracked.
This can give you an idea from where we can start: https://lkml.org/lkml/2011/3/14/346
Jiri, would you be interested in working on this?
yep, I remember also talking about A/B being functions entry/exit
I think that was also the initial push for having ftrace perf events support, which already got in.
I'll check ;)
jirka
Re: [RFD] perf: events defined contexts (was Re: perf: prctl(PR_TASK_PERF_EVENTS_DISABLE) has no effect)
Hi,
Quoted 28 lines · Jiri Olsa
Quoted 27 lines · Frederic Weisbecker
The problem is more general than that I think.
We need to be able to define finer grained contexts than just "task" and/or "CPU".And reusing events themselves would be a nice interface.
For example create 3 events:
A = irq entry tracepoint
B = irq exit tracepoint
C = cpu-cyclesAnd say: I want to count cpu-cycles when event A fires and stop counting when B fires.
With that you can count cpu cycles on irqs.
You could use any event you want to define your contexts: lock, functions, etc...
And even uprobes to define areas in userspace to profile. Would that solve the initial problem in the thread? Like hook on function library entry/exit?
I talked about that to Jiri Olsa several times. May be he would be interested in implementing this. I posted some patchets one year ago but got sidetracked.
This can give you an idea from where we can start: https://lkml.org/lkml/2011/3/14/346
Quoted 9 lines
Quoted 1 line
Jiri, would you be interested in working on this?
yep, I remember also talking about A/B being functions entry/exit
I think that was also the initial push for having ftrace perf events support, which already got in.
I'll check ;)
Very cool. :) I'm also interested in the feature although my background knowledge is not firm enough yet. Please let me know if there's anything I can help you on this.
Thanks,
Namhyung