Sample memory references with at least a given latency?
Hello list,
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).
Thank you very much in advance.
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Hello Manuel,
that's a very nice interesting piece of code! Thank you very much!
Now two different questions arise:
* still, is this possible through regular perf binary from the
linux kernel?
* with respect to your code, I observe that the data region for the
PEBS collected samples is pointed by metadata_page. However, when printing its contents through print_samples the output does not print time-stamps. Are the time-stamps collected? If so, how can one access to that information?
Thank you very much in advance!!
Quoted 30 lines · Manuel Selva
Yes you can. I wrote a small "memory not" for how to do this while trying to enable memory sampling:
https://manuelselva.wordpress.com/2013/10/31/how-to-use-the-linux-perf_event_open-system-call/
I also wrote some code exampl here:
https://github.com/ManuelSelva/c4fun/tree/master/pebs_tests2015-02-04 13:48 GMT+01:00 Harald Servat <[email protected] <mailto:[email protected]>>:
Hello list,
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).Thank you very much in advance.
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Quoted 10 lines
http://www.bsc.es/disclaimer
--
To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/__majordomo-info.html
<http://vger.kernel.org/majordomo-info.html>
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Hello Manuel,
I'm running Linux 3.11 but I don't see a way to setup the latency
threshold there. My option flags include:
-t, --type=
-D, --dump-raw-samples=
-x, --field-separator
-C, --cpu-list so I guess that this option is not implemented in perf (at least on
3.11). So I need to capture all the samples and then discard those that surpass that threshold.
Going back into your example, is it possible to capture / show the
timestamp for the captured samples? I've seen the Intel documentation, and there aren't timestamps in PEBS entries so that may require additional work. IIRC, someone of the list told me that perf allocates a 1-entry PEBS buffer so every time PEBS fills it, perf somehow gets an interrupt indicating that it has to flush it. This way, perf can attribute a timestamp to that sample also.
Thank you very much!
Quoted 34 lines · Manuel Selva
Hi Harald,
Yes you can with the "perf mem" tool. It has been introduced in kernel 3.10 (correct me if I am wrong, not very sure about that). It's built on top of perf mem record and perf mem report but hide for you the complxity of choosing the memory events . See "man perf mem" for th details.
----------
Manuel2015-02-04 13:48 GMT+01:00 Harald Servat <[email protected] <mailto:[email protected]>>:
Hello list,
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).Thank you very much in advance.
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Quoted 10 lines
http://www.bsc.es/disclaimer
--
To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/__majordomo-info.html
<http://vger.kernel.org/majordomo-info.html>
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Quoted 11 lines · Harald Servat
I'm running Linux 3.11 but I don't see a way to setup the latency threshold
there. My option flags include:-t, --type= -D, --dump-raw-samples= -x, --field-separator -C, --cpu-listso I guess that this option is not implemented in perf (at least on 3.11).
So I need to capture all the samples and then discard those that surpass that threshold.
No, PEBS support is very lacking with perf.
The soon-to-be-released 3.19 kernel does add support for PERF_SAMPLE_REGS_INTR which finally lets you get the PEBS register state from the PEBS sample, but I think many of the other advanced PEBS features are still missing.
Vince
Hi Harald,
You are right, the perf mem tool dosn't support stting the latency threshold. It should be a very easy first contribution to kernel for myself ;-)
Regarding the timestamp, I nver need such information, but as you said you can't tll to the hardware (on Intel I mean) to record it along with processor state information. I don't know how th kernel part of perf configure PEBS, but why would the kernel pay the overhead of an interrupt by using 1 slot buffers ? Only for recording timestamp ?
----
Manu
Quoted 86 lines · Harald Servat
Hello Manuel,
I'm running Linux 3.11 but I don't see a way to setup the latency
threshold there. My option flags include:-t, --type= -D, --dump-raw-samples= -x, --field-separator -C, --cpu-listso I guess that this option is not implemented in perf (at least on
3.11). So I need to capture all the samples and then discard those that surpass that threshold.Going back into your example, is it possible to capture / show the
timestamp for the captured samples? I've seen the Intel documentation, and there aren't timestamps in PEBS entries so that may require additional work. IIRC, someone of the list told me that perf allocates a 1-entry PEBS buffer so every time PEBS fills it, perf somehow gets an interrupt indicating that it has to flush it. This way, perf can attribute a timestamp to that sample also.Thank you very much!
Quoted 36 lines · Manuel Selva
Hi Harald,
Yes you can with the "perf mem" tool. It has been introduced in kernel 3.10 (correct me if I am wrong, not very sure about that). It's built on top of perf mem record and perf mem report but hide for you the complxity of choosing the memory events . See "man perf mem" for th details.
----------
Manuel2015-02-04 13:48 GMT+01:00 Harald Servat <[email protected] <mailto:[email protected]>>:
Hello list,
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).Thank you very much in advance.
WARNING / LEGAL TEXT: This message is intended only for the use of
the
individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.Quoted 10 lines
http://www.bsc.es/disclaimer
--
To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/__majordomo-info.html
<http://vger.kernel.org/majordomo-info.html>WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Hello,
Quoted 5 lines · Manuel Selva
Hi Harald,
You are right, the perf mem tool dosn't support stting the latency threshold. It should be a very easy first contribution to kernel for myself ;-)
that would be great. If I had some spare time & a machine to mess a
bit, I'd gladly help you. BTW, do you know if there's any alternative for developing the kernel?
Quoted 5 lines
Regarding the timestamp, I nver need such information, but as you said you can't tll to the hardware (on Intel I mean) to record it along with processor state information. I don't know how th kernel part of perf configure PEBS, but why would the kernel pay the overhead of an interrupt by using 1 slot buffers ? Only for recording timestamp ?
Personally, I play with time-stamped trace-files and find capturing
the time-stamp extremely important! Andi Kleen (also from this list) confirmed this (1 slot buffer) but he didn't mention the exact reason. Still, and let me insist, this is perfect for my purposes so I'm ok with it. See previous mails from this ML:
http://article.gmane.org/gmane.linux.kernel.perf.user/1398/match=information+regarding+perf+tool
http://article.gmane.org/gmane.linux.kernel.perf.user/1375/match=information+regarding+perf+tool
Regards.
Quoted 91 lines
----
ManuQuoted 87 lines · Harald Servat
Hello Manuel,
I'm running Linux 3.11 but I don't see a way to setup the latency
threshold there. My option flags include:-t, --type= -D, --dump-raw-samples= -x, --field-separator -C, --cpu-listso I guess that this option is not implemented in perf (at least on
3.11). So I need to capture all the samples and then discard those that surpass that threshold.Going back into your example, is it possible to capture / show the
timestamp for the captured samples? I've seen the Intel documentation, and there aren't timestamps in PEBS entries so that may require additional work. IIRC, someone of the list told me that perf allocates a 1-entry PEBS buffer so every time PEBS fills it, perf somehow gets an interrupt indicating that it has to flush it. This way, perf can attribute a timestamp to that sample also.Thank you very much!
Quoted 38 lines · Manuel Selva
Hi Harald,
Yes you can with the "perf mem" tool. It has been introduced in kernel 3.10 (correct me if I am wrong, not very sure about that). It's built on top of perf mem record and perf mem report but hide for you the complxity of choosing the memory events . See "man perf mem" for th details.
----------
Manuel2015-02-04 13:48 GMT+01:00 Harald Servat <[email protected] <mailto:[email protected]>>:
Hello list,
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).Thank you very much in advance.
WARNING / LEGAL TEXT: This message is intended only for the use of
the
individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or
exempt
from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to
the
intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.Quoted 10 lines
http://www.bsc.es/disclaimer
--
To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/__majordomo-info.html
<http://vger.kernel.org/majordomo-info.html>WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Quoted 6 lines
Quoted 1 line
--
To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Quoted 4 lines · Harald Servat
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).
$ ocperf.py record -e mem_trans_retired.load_latency_gt_128 -a sleep 1 perf record -e cpu/event=0xcd,umask=0x1,ldlat=0x80,name=mem_trans_retired_load_latency_gt_128/ -a sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.438 MB perf.data (~19125 samples) ]
Signature · 1 line
[email protected] -- Speaking for myself only
What information is present in the samples recorded by perf record (on which ocperf relies) ?
Manu
Quoted 14 lines · Andi Kleen
Quoted 4 lines · Harald Servat
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).$ ocperf.py record -e mem_trans_retired.load_latency_gt_128 -a sleep 1 perf record -e cpu/event=0xcd,umask=0x1,ldlat=0x80,name=mem_trans_retired_load_latency_gt_128/ -a sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.438 MB perf.data (~19125 samples) ]
Quoted 2 lines · Manuel Selva
What information is present in the samples recorded by perf record (on which ocperf relies) ?
man perf_event_open
man perf-record
It depends on what options you pass to record. As of the latest kernel practically everything in PEBS is supported. Before that the GPRs were missing. You can get for example memory source, address or TSX information.
-Andi
Hello Manuel, et al,
I've been digging a bit into the example code you mention below. It
look like lines 246-271 setup the PEBS infrastructure given through perf.
I notice, and correct if I'm wrong, that you don't setup any
interrupt service, so you expect that the buffer for the PEBS entries is sufficiently large for the application run. Do you know how to program an interrupt service if the buffer cannot hold all the PEBS entries? And is that possible from user-space?
Best regards and thank you.
Quoted 30 lines · Manuel Selva
Yes you can. I wrote a small "memory not" for how to do this while trying to enable memory sampling:
https://manuelselva.wordpress.com/2013/10/31/how-to-use-the-linux-perf_event_open-system-call/
I also wrote some code exampl here:
https://github.com/ManuelSelva/c4fun/tree/master/pebs_tests2015-02-04 13:48 GMT+01:00 Harald Servat <[email protected] <mailto:[email protected]>>:
Hello list,
is there any way to configure perf to sample memory references
through PEBS and enable the Load Latency Performance Monitoring Facility (section 18.7.1.2 from the Intel 64 and IA-32 Architectures Software Developer's Manual).Thank you very much in advance.
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.
Quoted 10 lines
http://www.bsc.es/disclaimer
--
To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
the body of a message to [email protected] <mailto:[email protected]>
More majordomo info at http://vger.kernel.org/__majordomo-info.html
<http://vger.kernel.org/majordomo-info.html>
WARNING / LEGAL TEXT: This message is intended only for the use of the individual or entity to which it is addressed and may contain information which is privileged, confidential, proprietary, or exempt from disclosure under applicable law. If you are not the intended recipient or the person responsible for delivering the message to the intended recipient, you are strictly prohibited from disclosing, distributing, copying, or in any way using this message. If you have received this communication in error, please notify the sender and destroy and delete any copies you may have received.