Skip to contentSkip to conversation

perf_disable()

DISCUSSION Peter Zijlstra 4 participants 8 messages Jun 11, 2010 – Jun 14, 2010

lore
Share & tools
lore
#0 lore raw

Hi,

I've been going over perf_disable() usage in kernel/perf_event.c and wondered if we actually need it at all.

Currently the only thing we seem to require it for is around pmu::enable calls (and for that powerpc at least does it itself, on x86 we rely on it to call ->enable_all and reprogram the pmu state).

But I can't really find any NMI races wrt data structures or the like as seems implied by some comments.

There is a fun little recursion issue with perf_adjust_period(), where if we fully removed perf_disable() we could end up calling pmu::stop() twice and such.

But aside from that it looks to me its mostly about optimizing hardware writes.

If nobody else known about/can find anything, I'm going to mostly remove perf_disable() for now and later think about how to optimize the hardware writes again.

#1 lore raw
Quoted 9 lines · Peter Zijlstra

I've been going over perf_disable() usage in kernel/perf_event.c and wondered if we actually need it at all.

Currently the only thing we seem to require it for is around pmu::enable calls (and for that powerpc at least does it itself, on x86 we rely on it to call ->enable_all and reprogram the pmu state).

But I can't really find any NMI races wrt data structures or the like as seems implied by some comments.

Yes, it was originally used to disable nmis for some critical sections in the non-arch code. I do not remember where this was exactly needed, but my feeling is also this can be optimized and maybe reimplemented as non-locking code.

We also should avoid the enable_all/disable_all() functions in the x86 implementation as it is expensive on some pmus (namely AMD). It looks like these functions can be removed then too, or at least made model specific only.

-Robert

Signature · 3 lines
Advanced Micro Devices, Inc.
Operating System Research Center
email: [email protected]
#2 lore raw
In reply to Peter Zijlstra
Quoted 11 lines · Peter Zijlstra

Hi,

I've been going over perf_disable() usage in kernel/perf_event.c and wondered if we actually need it at all.

Currently the only thing we seem to require it for is around pmu::enable calls (and for that powerpc at least does it itself, on x86 we rely on it to call ->enable_all and reprogram the pmu state).

But I can't really find any NMI races wrt data structures or the like as seems implied by some comments.

I suspect the problem is also on per context integrity. When you adjust the period, enable or disable a counter, this counter becomes async with the rest of the group or the rest of the counters in the same context, for a small bunch of time.

The longer you run your events, the higher is going to be this jitter.

Take an example, when you adjust a period, you:

perf_disable()
perf_event_stop()
left_period = 0
perf_event_start()
perf_enable()

During all this time, the given event is paused, but the whole rest of the events running on the cpu continue to count.

The problem is the same on context switch.

And I think this high resolution of synchronisation per context is sensitive, especially with perf start kind of workflows.

(Although software events are not touched by perf_enable()/perf_disable().

Quoted 11 lines

There is a fun little recursion issue with perf_adjust_period(), where if we fully removed perf_disable() we could end up calling pmu::stop() twice and such.

But aside from that it looks to me its mostly about optimizing hardware writes.

If nobody else known about/can find anything, I'm going to mostly remove perf_disable() for now and later think about how to optimize the hardware writes again.

Not sure that's a good idea IMHO.

#3 lore raw
Quoted 30 lines · Frederic Weisbecker
Quoted 11 lines · Peter Zijlstra

Hi,

I've been going over perf_disable() usage in kernel/perf_event.c and wondered if we actually need it at all.

Currently the only thing we seem to require it for is around pmu::enable calls (and for that powerpc at least does it itself, on x86 we rely on it to call ->enable_all and reprogram the pmu state).

But I can't really find any NMI races wrt data structures or the like as seems implied by some comments.

I suspect the problem is also on per context integrity. When you adjust the period, enable or disable a counter, this counter becomes async with the rest of the group or the rest of the counters in the same context, for a small bunch of time.

The longer you run your events, the higher is going to be this jitter.

Take an example, when you adjust a period, you:

perf_disable()
perf_event_stop()
left_period = 0
perf_event_start()
perf_enable()

Quoted 7 lines

During all this time, the given event is paused, but the whole rest of the events running on the cpu continue to count.

The problem is the same on context switch.

And I think this high resolution of synchronisation per context is sensitive, especially with perf start kind of workflows.

I'm not sure what you're arguing, but the knife cuts on both sides, the above also stops counters that shouldn't be stopped..

Quoted 7 lines
Quoted 7 lines

There is a fun little recursion issue with perf_adjust_period(), where if we fully removed perf_disable() we could end up calling pmu::stop() twice and such.

But aside from that it looks to me its mostly about optimizing hardware writes.

Quoted 6 lines
Quoted 3 lines

If nobody else known about/can find anything, I'm going to mostly remove perf_disable() for now and later think about how to optimize the hardware writes again.

Not sure that's a good idea IMHO.

Well, we need to do something, the current weak mess needs to go, and the alternative is basically a loop over all registerd pmus calling their respective pmu::disable_all, which is utter suckage, so removing as many of this as possible is a good thing.

We can always come up with some lazy mode later that tries to batch the hardware writes.

#4 lore raw
Quoted 19 lines · Peter Zijlstra
Quoted 18 lines · Frederic Weisbecker

I suspect the problem is also on per context integrity. When you adjust the period, enable or disable a counter, this counter becomes async with the rest of the group or the rest of the counters in the same context, for a small bunch of time.

The longer you run your events, the higher is going to be this jitter.

Take an example, when you adjust a period, you:

perf_disable()
perf_event_stop()
left_period = 0
perf_event_start()
perf_enable()

During all this time, the given event is paused, but the whole rest of the events running on the cpu continue to count.

Quoted 7 lines
Quoted 4 lines

The problem is the same on context switch.

And I think this high resolution of synchronisation per context is sensitive, especially with perf start kind of workflows.

I'm not sure what you're arguing, but the knife cuts on both sides, the above also stops counters that shouldn't be stopped..

Hmm, now that I look at it, x86_pmu_*able_all() isn't touching a single register to *able everything at once, it's doing a loop on every events.

Forget about what I said then.

Quoted 14 lines
Quoted 7 lines
Quoted 7 lines

There is a fun little recursion issue with perf_adjust_period(), where if we fully removed perf_disable() we could end up calling pmu::stop() twice and such.

But aside from that it looks to me its mostly about optimizing hardware writes.

Quoted 6 lines
Quoted 3 lines

If nobody else known about/can find anything, I'm going to mostly remove perf_disable() for now and later think about how to optimize the hardware writes again.

Not sure that's a good idea IMHO.

Quoted 7 lines

Well, we need to do something, the current weak mess needs to go, and the alternative is basically a loop over all registerd pmus calling their respective pmu::disable_all, which is utter suckage, so removing as many of this as possible is a good thing.

We can always come up with some lazy mode later that tries to batch the hardware writes.

Right.

#5 lore raw
In reply to Peter Zijlstra
Quoted 3 lines · Peter Zijlstra

There is a fun little recursion issue with perf_adjust_period(), where if we fully removed perf_disable() we could end up calling pmu::stop() twice and such.

We can have a local_t made nesting level on the stop/start that could easily deal with this.

#6 lore raw
Quoted 9 lines · Frederic Weisbecker
Quoted 3 lines · Peter Zijlstra

There is a fun little recursion issue with perf_adjust_period(), where if we fully removed perf_disable() we could end up calling pmu::stop() twice and such.

We can have a local_t made nesting level on the stop/start that could easily deal with this.

Fanning it out to the relevant pmu implementation is the interesting part..

#7 lore raw
In reply to Peter Zijlstra

Hi Peter,

Quoted 4 lines · Peter Zijlstra

Hi,

I've been going over perf_disable() usage in kernel/perf_event.c and wondered if we actually need it at all.

-------->8--------

Quoted 3 lines

If nobody else known about/can find anything, I'm going to mostly remove perf_disable() for now and later think about how to optimize the hardware writes again.

No objections from the ARM camp as we don't use perf_disable() for anything anyway.

Will