git bisect and perf

I am trying find out when (ie. which commit) the pebs feature actually started working on a westmere system and hitting a problem with git-bisect:

# git bisect start arch/x86/kernel/cpu/perf_event*

# git bisect bad v2.6.35

# git bisect good v2.6.38
Some good revs are not ancestor of the bad rev.
git bisect cannot work properly in this case.
Maybe you mistake good and bad revs?

Any ideas on why git bisect would not work?

David

git bisect is telling you what is wrong -- as the man page says:

This command uses git rev-list --bisect to help drive the binary search process to find which change introduced a bug, given an old "good" commit object name and a later "bad" commit object name.

so it assumes the good commit is older than the bad commit.

You can actually use git bisect in your case, although it gets very confusing unless you write yourself a little wrapper alias: just swap the meaning of good and bad, ie do

# git bisect start arch/x86/kernel/cpu/perf_event*
# git bisect good v2.6.35
# git bisect bad v2.6.38

and try somehow to remember that when you do a test, "bad" means PEBS works and "good" means PEBS doesn't work.

- R.

What do you mean with working? The whole cycles:pp thing is magic and unrelated to 'regular' PEBS stuff. .35 would indeed be the version we introduced PEBS support and it should actually work.

But if you're looking for the cycles:pp stuff (commit 7639dae0ca1) that's .38-rc1-ish.

So on kernels pre that commit you could still do:

perf record -e r108000c0:pp

and get the same effect. Its just that after that commit r3c:pp will be magically translated to the above.

And just to be complete (as Stephane always needs to point out) event 3c and 108000c0 aren't fully identical, they differ in that one counts while idle and the other doesn't (but I forgot which way around).

Luckily not a lot of people care about profiling idle time.

Thanks, Roland. I'll reverse the logic.

David

:p requires PEBS?

I'm referring to cycles:p

model name : Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
Fedora 14, 2.6.35.14-106.fc14.x86_64 (few perf_event* differences from 2.6.35.14).

# perf record -e cycles:p -v -- sleep 1

Warning: ... trying to fall back to cpu-clock-ticks

...

Cycles is supported; it's the :p that is not. perf userspace only recently gained the knowledge for discriminating the two. Using perf from acme/perf/urgent:

# /tmp/pbuild/perf record -e cycles:p -v -- sleep 1

Error: sys_perf_event_open() syscall returned with 28 (No space left
on device). /bin/dmesg may provide additional information.

I chased this to intel_pebs_constraints() returning &emptyconstraint which has a weight of 0 (validate_event -> intel_get_event_constraints -> intel_pebs_constraints).

Indeed that works on the F14 kernel.

David

On Intel it does.. On AMD you want IBS for that.

Note that on Intel there's a difference between :p and :pp.

/*
* precise_ip:
*
* 0 - SAMPLE_IP can have arbitrary skid
* 1 - SAMPLE_IP must have constant skid
* 2 - SAMPLE_IP requested to have 0 skid
* 3 - SAMPLE_IP must have 0 skid *
* See also PERF_RECORD_MISC_EXACT_IP */
precise_ip : 2, /* skid constraint */

So 0 being the default (no :p) we use the regular PMI and provide the IP from the spot that served the exception. This can be a few instructions _after_ the instruction that triggered the event overflow.

1 (:p) means constant skid, for Intel this maps to regular PEBS and returns the IP as provided by the PEBS record. This is one instruction after the instruction that caused the event overflow (because the PEBS hardware assist triggers when we're executing the instruction that triggered it, it needs to have already fetched the instruction and incremented the IP etc..).

2 (:pp) requests 0 skid, since PEBS provides IP + 1 instruction, we need to rewind the instruction stream. In order to do this we use the LBR in conjunction with a FREEZE_LBR_ON_PMI feature from DEBUG_CTL and limit the PEBS buffer to a single entry. So when we get the PEBS buffer full PMI, we read the LBR, find the start of the most recent basic block (to address of the last branch) and decode the instruction stream until we find the current IP. Then we know the IP that caused it is the previous one. Tada!

We set the PERF_RECORD_MISC_EXACT_IP bit when the rewind worked and we can guarantee 0 skid (+- hardware bugs, of which there are a few).

3 (:ppp) would require 0 skid, but since the unwind using LBR isn't a guaranteed thing we don't implement this for Intel (or anybody else for that matter afaik).

Now not all events the Intel PMU can do are PEBS capable; in fact most aren't. So the whole cycles thing is about mapping the regular cycles event to something that is PEBS capable.

That's not as helpful as /proc/cpuinfo. I suspect this is a Westmere system, isn't it?

Yes:
processor : 15
vendor_id : GenuineIntel
cpu family : 6
model : 44
model name : Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
stepping : 2
cpu MHz : 2394.133
cache size : 12288 KB

[    0.146055] Performance Events: PEBS fmt1+, Westmere events, Intel PMU driver.
[    0.146645] ... version:                3
[    0.146897] ... bit width:              48
[    0.147150] ... generic registers:      4
[    0.147404] ... value mask:             0000ffffffffffff
[    0.147657] ... max period:             000000007fffffff
[    0.147907] ... fixed-purpose events:   3
[    0.148158] ... event mask:             000000070000000f

Thanks for the explanation and patience.

I think I get it now. 7639dae0 is the commit I needed. With it 2.6.35.14 can do cycles:pp profiling.

In general only the events listed in intel_XXXXX_pebs_events can be used with :p.

David

Yes.