Interrupt caused by software events

Hello,

I'm learning the perf_events programming interface and I'm not sure if what I'm trying to do is possible. I could only find the perf_event_open documentation, but unfortunately that's not enough for my needs, as it only counts the number of events that happened.

I'm looking for a way to determine which instruction caused some events:
something similar to what perf annotate does, but in an online fashion. Ideally, I'd like to have some kind of handler that gets called whenever a perf event occurs. Is that even possible? If not, is there any way I could get this information in a real-time way?

I'll be using PEBS or IBS events.

Thank you,
-Martin

hi,
you could use perf_event_create_kernel_counter interface, check the lockup watchdog in kernel/watchdog.c

jirka

Vince and Jiri,

Thank you very much for your replies.

I re-read my email and I'm not sure I was clear, so just in case I didn't made myself clear, here's a high-level example of what I'm looking for.

/*******************/
/* Gets called whenever my program generates a cache-miss */ void my_func()

{
    unsigned long addr = get_ip_that_generated_event();
    printf("A cache-miss occurred at IP: %x!\n", addr);
}

int main()

{
    struct perf_event_attr pe;
    pe.type = PERF_TYPE_HARDWARE.
    pe.config = PERF_COUNT_HW_CACHE_MISSES;
    ...
    register_handler(&pe, my_func);
    // code to be monitored
    unregister_handler();
    return 0;
}

/*******************/

Every time an event occurs (a cache-miss in my example), my_func should get called. The get_ip_that_generated_event() function should return the Instruction Pointer that generated the event.

My impression is that this could be done with a low overhead (other then that introduced by what I do in my_func), because
"perf record -e cache-misses ./a.out"
can pinpoint with little overhead how many times the event was caused by each instruction. But maybe I'm just missing something.

Vince:
I'm unsure how to configure the mmap buffer you mentioned, and I couldn't find any examples online. Could you please point me to some reference, or maybe provide a little example?

Jiri:
I'm under the impression I'd have to be in a kernel module (or in kernel mode in general) to use perf_event_create_kernel_counter(). Is that correct?

Thank you once again,
-Martin

In general you can't do this. Cache misses happen way too often, your program would not be able to keep up, and likely it would be causing cache misses itself.

Also you should read up on skid, it's very hard to track an event to an exact IP, although if your CPU has PEBS it's possible with certain limitations.

No, what perf record is doing is sampling. It's getting interrupted say every 100,000 cache misses and reporting the values, and hopefully the ones that trigger are statistically relevant enough to extract the full program behavior.

There are some examples in my perf_event_tests package.

See for example
https://github.com/deater/perf_event_tests/blob/master/tests/x86_intel/pebs.c

although that example really isn't using PEBS yet, I'm currently working on that code, so beware I might be re-writing and moving around the code in that directory in the next few days.

Vince

SNIP

yep, thats kernel space only

jirka

This will almost certainly not work with any useful workload. Just compare the overhead of the instrumentation with the total run time. The program will be doing nothing but such interrupts and perf will eventually throttle the interrupts. Besides PEBS is not designed to catch every event anyways.

If you really want to instrument all cache misses you most likely want to use some memory trace mechanism and a offline cache model or a simulator.

-Andi

[email protected] -- Speaking for myself only

Not really sure by what you mean by "real-time", but you can theoretically set up the perf_event mmap buffer with a low threshold, and have it configured to send a signal to your program whenever an event comes in. In theory you can get the PEBS-like info out that way. It's going to have fairly high overhead though, especially if you set the sample rate high.

It's even worse if you are trying to get full info on *every* event. That's probably not possible with any even that happens more than a few times a second.

Vince