perf: Support PERF_SAMPLE_READ with inherit_stat
v4 cover perf: Support PERF_SAMPLE_READ with inherit_stat
This change allows events to use PERF_SAMPLE READ with inherit so long as both inherit_stat and PERF_SAMPLE_TID are set.
Currently it is not possible to use PERF_SAMPLE_READ with inherit. This restriction assumes the user is interested in collecting aggregate statistics as per perf stat. It prevents a user from collecting per-thread samples using counter groups from a multi-threaded or multi-process application, as with perf record -e '{....}:S'. Instead users must use system-wide mode, or forgo the ability to sample counter groups. System-wide mode is often problematic as it requires specific permissions (no CAP_PERFMON / root access), or may lead to capture of significant amounts of extra data from other processes running on the system.
Perf already supports the ability to collect per-thread counts with inherit via the inherit_stat flag. This patch changes perf_event_alloc relaxing the restriction to combine inherit with PERF_SAMPLE_READ so that the combination will be allowed so long as inherit_stat and PERF_SAMPLE_TID are enabled.
In this configuration stream ids (such as may appear in the read_format field of a PERF_RECORD_SAMPLE) are no longer globally unique, rather the pair of (stream id, tid) uniquely identify each event. Tools that rely on this, for example to calculate a delta between samples, would need updating to take this into account. Previously valid event configurations (system-wide, no-inherit and so on) where each stream id is the identifier are unaffected.
Changes since v3:
- Cleaned up perf test data changes incorrectly included into this series from elsewhere.
Changes since v2:
- Rebase on v6.8
- Respond to James Clarke's feedback; fixup some typos and move some repeated checks into a helper macro.
- Cleaned up checkpatch lints.
- Updated perf test; fixed evsel handling so that existing tests pass and added new tests to cover the new behaviour.
Changes since v1:
- Rebase on v6.8-rc1
- Fixed value written into sample after child exists.
- Modified handling of switch-out so that context with these events take the slow path, so that the per-event/per-thread PMU state is correctly switched.
- Modified perf tools to support this mode of operation.
Ben Gainey (4):
perf: Support PERF_SAMPLE_READ with inherit_stat tools/perf: Track where perf_sample_ids need per-thread periods tools/perf: Correctly calculate sample period for inherited
SAMPLE_READ values
tools/perf: Allow inherit + inherit_stat + PERF_SAMPLE_READ when
opening events
include/linux/perf_event.h | 1 + kernel/events/core.c | 62 ++++++++++---- tools/lib/perf/evlist.c | 1 + tools/lib/perf/evsel.c | 48 +++++++++++ tools/lib/perf/include/internal/evsel.h | 55 ++++++++++++- .../test-record-group-sampling-inherit-stat | 62 ++++++++++++++ tools/perf/util/evsel.c | 82 ++++++++++++++++++- tools/perf/util/evsel.h | 1 + tools/perf/util/session.c | 11 ++- 9 files changed, 301 insertions(+), 22 deletions(-) create mode 100644 tools/perf/tests/attr/test-record-group-sampling-inherit-stat
Signature · 1 line
2.44.0
PATCH v4 1/4 perf: Support PERF_SAMPLE_READ with inherit_stat
This change allows events to use PERF_SAMPLE READ with inherit so long as both inherit_stat and PERF_SAMPLE_TID are set.
In this configuration, an event will be inherited into any child processes / threads, allowing convenient profiling of a multiprocess or multithreaded application, whilst allowing profiling tools to collect per-thread samples, in particular of groups of counters.
--- include/linux/perf_event.h | 1 + kernel/events/core.c | 62 +++++++++++++++++++++++++++++--------- 2 files changed, 48 insertions(+), 15 deletions(-)
include/linux/perf_event.h +1 -0
@@ -932,6 +932,7 @@ struct perf_event_context {int nr_task_data;int nr_stat; +int nr_stat_read;int nr_freq;int rotate_disable;
kernel/events/core.c +47 -15
161 lines not on this page · raw message
Signature · 1 line
2.44.0
PATCH v4 2/4 tools/perf: Track where perf_sample_ids need per-thread periods
In reply to Ben Gaineyperf_sample_ids and related evlist/evsel code are modified to track which events combine inherit+inherit_stat+PERF_SAMPLE_READ.
Events with this combination of properties must be handled differently when tracking each streams sample period, as their stream-id is now only unique per thread, rather than globally unique.
--- tools/lib/perf/evlist.c | 1 + tools/lib/perf/evsel.c | 7 +++++++ tools/lib/perf/include/internal/evsel.h | 9 +++++++++ 3 files changed, 17 insertions(+)
tools/lib/perf/evlist.c +1 -0
@@ -255,6 +255,7 @@ static void perf_evlist__id_hash(struct perf_evlist *evlist, sid->id = id; sid->evsel = evsel; + sid->period_per_thread =perf_evsel__attr_has_per_thread_sample_period (evsel); hash =hash_64 (sid->id,PERF_EVLIST__HLIST_BITS );hlist_add_head (&sid->node, &evlist->heads[hash]); }
tools/lib/perf/evsel.c +7 -0
@@ -537,6 +537,13 @@ void perf_evsel__free_id(struct perf_evsel *evsel) evsel->ids =0 ; } +bool perf_evsel__attr_has_per_thread_sample_period (struct perf_evsel *evsel) +{ +return (evsel->attr.sample_type &PERF_SAMPLE_READ ) + && evsel->attr.inherit + && evsel->attr.inherit_stat; +} +void perf_counts_values__scale (struct perf_counts_values *count,bool scale,__s8 *pscaled) {
tools/lib/perf/include/internal/evsel.h +9 -0
@@ -36,6 +36,13 @@ struct perf_sample_id {/* Holds total ID period value for PERF_SAMPLE_READ processing. */ u64 period; + +/* +* When inherit+inherit_stat is combined with PERF_SAMPLE_READ, the +* period value is per (sample_id, thread) tuple, rather than per +* sample_id. +*/ +bool period_per_thread; };struct perf_evsel {
@@ -88,4 +95,6 @@ int perf_evsel__apply_filter(struct perf_evsel *evsel, const char *filter);int perf_evsel__alloc_id (struct perf_evsel *evsel,int ncpus,int nthreads);void perf_evsel__free_id (struct perf_evsel *evsel); +bool perf_evsel__attr_has_per_thread_sample_period (struct perf_evsel *evsel); +#endif /* __LIBPERF_INTERNAL_EVSEL_H */
Signature · 1 line
2.44.0
PATCH v4 3/4 tools/perf: Correctly calculate sample period for inherited SAMPLE_READ values
In reply to Ben GaineySample period calculation is updated to take into account the fact that the stream-id for events with inherit+inherit_stat+PERF_SAMPLE_READ is now unique per thread, rather than globally unique.
perf_sample_id is modified to support tracking per-stream+per-thread values, along with the existing global per-stream values. In the per-thread case, values are stored in a hash by tid.
deliver_sample_value is modified to correctly locate the previous sample storage based on the attribute, stream id and thread id.
--- tools/lib/perf/evsel.c | 41 ++++++++++++++++++++++ tools/lib/perf/include/internal/evsel.h | 46 +++++++++++++++++++++++-- tools/perf/util/session.c | 11 ++++-- 3 files changed, 93 insertions(+), 5 deletions(-)
tools/lib/perf/evsel.c +41 -0
@@ -5,6 +5,7 @@#include <perf/evsel.h> #include <perf/cpumap.h> #include <perf/threadmap.h> +#include <linux/hash.h> #include <linux/list.h> #include <internal/evsel.h> #include <linux/zalloc.h>
@@ -23,6 +24,7 @@ void perf_evsel__init(struct perf_evsel *evsel, struct perf_event_attr *attr, int idx) {INIT_LIST_HEAD (&evsel->node); +INIT_LIST_HEAD (&evsel->period_per_thread_periods); evsel->attr = *attr; evsel->idx = idx; evsel->leader = evsel;
@@ -531,10 +533,17 @@ int perf_evsel__alloc_id(struct perf_evsel *evsel, int ncpus, int nthreads)void perf_evsel__free_id (struct perf_evsel *evsel) { +struct perf_sample_id_period *pos, *n; +xyarray__delete (evsel->sample_id); evsel->sample_id =NULL ;zfree (&evsel->id); evsel->ids =0 ; + +perf_evsel_for_each_per_thread_period_safe (evsel, n, pos) { +list_del_init (&pos->node); +zfree (pos); + } }bool perf_evsel__attr_has_per_thread_sample_period (struct perf_evsel *evsel)
@@ -544,6 +553,38 @@ bool perf_evsel__attr_has_per_thread_sample_period(struct perf_evsel *evsel) && evsel->attr.inherit_stat; } +u64 *perf_sample_id__get_period_storage (struct perf_sample_id *sid,u32 tid) +{ +struct hlist_head *head; +struct perf_sample_id_period *res; +int hash; + +if (!sid->period_per_thread) +return &sid->period; + + hash =hash_32 (tid,PERF_SAMPLE_ID__HLIST_BITS ); + head = &sid->periods[hash]; + +hlist_for_each_entry (res, head, hnode) +if (res->tid == tid) +return &res->period; + +if (sid->evsel ==NULL ) +return NULL ; + + res =zalloc (sizeof (struct perf_sample_id_period )); +if (res ==NULL ) +return NULL ; + +INIT_LIST_HEAD (&res->node); + res->tid = tid; + +list_add_tail (&res->node, &sid->evsel->period_per_thread_periods); +hlist_add_head (&res->hnode, &sid->periods[hash]); + +return &res->period; +} +void perf_counts_values__scale (struct perf_counts_values *count,bool scale,__s8 *pscaled) {
tools/lib/perf/include/internal/evsel.h +44 -2
74 lines not on this page · raw message
tools/perf/util/session.c +8 -3
@@ -1478,14 +1478,19 @@ static int deliver_sample_value(struct evlist *evlist, {struct perf_sample_id *sid =evlist__id2sid (evlist, v->id);struct evsel *evsel; +u64 *storage =NULL ;if (sid) { + storage =perf_sample_id__get_period_storage (sid, sample->tid); + } + +if (storage) { sample->id = v->id; - sample->period = v->value - sid->period; - sid->period = v->value; + sample->period = v->value - *storage; + *storage = v->value; } -if (!sid || sid->evsel ==NULL ) { +if (!storage || sid->evsel ==NULL ) { ++evlist->stats.nr_unknown_id;return 0 ; }
Signature · 1 line
2.44.0
PATCH v4 4/4 tools/perf: Allow inherit + inherit_stat + PERF_SAMPLE_READ when opening events
In reply to Ben GaineyThis change updates evsel to allow the combination of inherit and PERF_SAMPLE_READ.
A fallback is implemented for kernel versions where this feature is not supported.
The user must pass --stat option to perf record to opt into this new behaviour.
--- .../test-record-group-sampling-inherit-stat | 62 ++++++++++++++ tools/perf/util/evsel.c | 82 ++++++++++++++++++- tools/perf/util/evsel.h | 1 + 3 files changed, 143 insertions(+), 2 deletions(-) create mode 100644 tools/perf/tests/attr/test-record-group-sampling-inherit-stat
tools/perf/tests/attr/test-record-group-sampling-inherit-stat new file +62 -0
@@ -0,0 +1,62 @@ +[config] +command = record +args = --stat --no-bpf-event -c 10000 -e '{cycles,cache-misses}:S' kill >/dev/null 2>&1 +ret = 1 + +[event-1:base-record] +fd=1 +group_fd=-1 + +# cycles +type=0 +config=0 + +# default | PERF_SAMPLE_READ +sample_type=87 + +# PERF_FORMAT_ID | PERF_FORMAT_GROUP | PERF_FORMAT_LOST | PERF_FORMAT_TOTAL_TIME_ENABLED | PERF_FORMAT_TOTAL_TIME_RUNNING +read_format=31 +task=1 +mmap=1 +comm=1 +enable_on_exec=1 +disabled=1 + +# inherit is enabled for group sampling, as is inherit_stat +inherit=1 +inherit_stat=1 + +# sampling disabled +sample_freq=0 +sample_period=10000 +freq=0 +write_backward=0 + +[event-2:base-record] +fd=2 +group_fd=1 + +# cache-misses +type=0 +config=3 + +# default | PERF_SAMPLE_READ +sample_type=87 + +# PERF_FORMAT_ID | PERF_FORMAT_GROUP | PERF_FORMAT_LOST | PERF_FORMAT_TOTAL_TIME_ENABLED | PERF_FORMAT_TOTAL_TIME_RUNNING +read_format=31 +task=0 +mmap=0 +comm=0 +enable_on_exec=0 +disabled=0 + +# inherit is enabled for group sampling, as is inherit_stat +inherit=1 +inherit_stat=1 + +# sampling disabled +sample_freq=0 +sample_period=0 +freq=0 +write_backward=0
tools/perf/util/evsel.c +80 -2
112 lines not on this page · raw message
tools/perf/util/evsel.h +1 -0
@@ -192,6 +192,7 @@ struct perf_missing_features {bool weight_struct;bool read_lost;bool branch_counters; +bool inherit_sample_read; };extern struct perf_missing_features perf_missing_features;
Signature · 1 line
2.44.0
Namhyung Kim
maintainerQuoted 15 lines · Ben Gainey
This change allows events to use
PERF_SAMPLEREAD with inherit so long as bothinherit_statandPERF_SAMPLE_TIDare set.Currently it is not possible to use
PERF_SAMPLE_READwith inherit. This restriction assumes the user is interested in collecting aggregate statistics as perperf stat. It prevents a user from collecting per-thread samples using counter groups from a multi-threaded or multi-process application, as withperf record -e '{....}:S'. Instead users must use system-wide mode, or forgo the ability to sample counter groups. System-wide mode is often problematic as it requires specific permissions (noCAP_PERFMON/ root access), or may lead to capture of significant amounts of extra data from other processes running on the system.
Quoted 2 lines
Perf already supports the ability to collect per-thread counts with
inheritvia theinherit_statflag. This patch changes
I'm not sure about this part. IIUC inherit and inherit_stat is not for per-thread counts, it only supports per-process (including children) events.
Quoted 3 lines
perf_event_allocrelaxing the restriction to combineinheritwithPERF_SAMPLE_READso that the combination will be allowed so long asinherit_statandPERF_SAMPLE_TIDare enabled.
Anyway, does it really need 'inherit_stat'? I think it's only for counting use cases (e.g. 'perf stat') not for sampling.
Also technically, it can have PERF_SAMPLE_STREAM_ID instead of PERF_SAMPLE_TID to distinguish the counter values.
Quoted 8 lines
In this configuration stream ids (such as may appear in the
read_formatfield of aPERF_RECORD_SAMPLE) are no longer globally unique, rather the pair of (stream id, tid) uniquely identify each event. Tools that rely on this, for example to calculate a delta between samples, would need updating to take this into account. Previously valid event configurations (system-wide, no-inherit and so on) where each stream id is the identifier are unaffected.
I think you meant PERF_SAMPLE_ID not PERF_SAMPLE_STREAM_ID. IIUC the stream id is already unique.
Thanks,
Namhyung
Quoted 45 lines
Changes since v3:
- Cleaned up perf test data changes incorrectly included into this series from elsewhere.Changes since v2:
- Rebase on v6.8
- Respond to James Clarke's feedback; fixup some typos and move some repeated checks into a helper macro.
- Cleaned up checkpatch lints.
- Updated perf test; fixed evsel handling so that existing tests pass and added new tests to cover the new behaviour.Changes since v1:
- Rebase on v6.8-rc1
- Fixed value written into sample after child exists.
- Modified handling of switch-out so that context with these events take the slow path, so that the per-event/per-thread PMU state is correctly switched.
- Modified perf tools to support this mode of operation.Ben Gainey (4):
perf: SupportPERF_SAMPLE_READwithinherit_stattools/perf: Track whereperf_sample_idsneed per-thread periodstools/perf: Correctly calculate sample period for inherited
SAMPLE_READvalues
tools/perf: Allow inherit +inherit_stat+PERF_SAMPLE_READwhen
opening eventsinclude/linux/perf_event.h | 1 + kernel/events/core.c | 62 ++++++++++---- tools/lib/perf/evlist.c | 1 + tools/lib/perf/evsel.c | 48 +++++++++++ tools/lib/perf/include/internal/evsel.h | 55 ++++++++++++- .../test-record-group-sampling-inherit-stat | 62 ++++++++++++++ tools/perf/util/evsel.c | 82 ++++++++++++++++++- tools/perf/util/evsel.h | 1 + tools/perf/util/session.c | 11 ++- 9 files changed, 301 insertions(+), 22 deletions(-) create mode 100644 tools/perf/tests/attr/test-record-group-sampling-inherit-stat--
2.44.0
Quoted 24 lines · Namhyung Kim
Quoted 22 lines · Ben Gainey
This change allows events to use
PERF_SAMPLEREAD with inherit so long
as bothinherit_statandPERF_SAMPLE_TIDare set.Currently it is not possible to use
PERF_SAMPLE_READwith inherit. This
restriction assumes the user is interested in collecting aggregate statistics as perperf stat. It prevents a user from collecting per-thread samples using counter groups from a multi-threaded or multi-process application, as withperf record -e '{....}:S'. Instead
users must use system-wide mode, or forgo the ability to sample counter
groups. System-wide mode is often problematic as it requires specific
permissions (noCAP_PERFMON/ root access), or may lead to capture of
significant amounts of extra data from other processes running on the
system.
Quoted 7 lines
Quoted 2 lines
Perf already supports the ability to collect per-thread counts with
inheritvia theinherit_statflag. This patch changesI'm not sure about this part. IIUC inherit and
inherit_statis not for
per-thread counts, it only supports per-process (including children) events.
Hi Namhyung
Thanks for the comments...
I don't think this is correct, if you compare the behaviour of
perf record --no-inherit ... <some-forking-processes> perf script -F pid,tid | sort -u
and
perf record --no-inherit ... <some-multithreaded-processes> perf script -F pid,tid | sort -u
vs
perf record ... <some-forking-processes>
perf script -F pid,tid | sort -u
and
perf record .. <some-multithreaded-processes>
perf script -F pid,tid | sort -u
The behaviour is consistent with the fact that no-inherit only records the primary thread of the primary process, whereas in the inherit case any child tasks (either threads or forked processes) is recorded.
Quoted 10 lines
Quoted 7 lines
perf_event_allocrelaxing the restriction to combineinheritwithPERF_SAMPLE_READso that the combination will be allowed so long asinherit_statandPERF_SAMPLE_TIDare enabled.Anyway, does it really need '
inherit_stat'? I think it's only for counting use cases (e.g. 'perf stat') not for sampling.
I would be very happy to remove the inherit_stat requirement. When I first came to this it seemed like the logic was all there in inherit_stat already, but now that I have to take a different path in perf_event_context_sched_out I suspect it should be trivial to remove the inherit_stat requirement.
Quoted 3 lines
Also technically, it can have
PERF_SAMPLE_STREAM_IDinstead ofPERF_SAMPLE_TIDto distinguish the counter values.
It looks like you are correct, but the ID given in the read_format part of PERF_SAMPLE_RECORD is the ID rather than STREAM_ID. (I had incorrectly thought/stated it was the latter). Hence when processing the read_format values in the sample record, we either need to use the TID to uniquely identify the source, or we would need to modify the read_format to (additionally) include the STREAM_ID.
* The current approach in tools uses the ID+TID, which puts more
complexity in the tools but means there isn't an extra field in the read_format data (for each value).
* Alternatively I could introduce a PERF_FORMAT_STREAM_ID; I would
expect that the user/tool would need to specify PERF_FORMAT_ID|PERF_FORMAT_STREAM_ID as they would need to use the ID to lookup the correct perf_event_attr, but could use the STREAM_ID to uniquely identify the child event. This approach would add an extra u64 per value in the read_format data but is possibly simpler/safer for tools?
Any preferences?
Quoted 15 lines
Quoted 14 lines
In this configuration stream ids (such as may appear in the
read_format
field of aPERF_RECORD_SAMPLE) are no longer globally unique, rather
the pair of (stream id, tid) uniquely identify each event. Tools that
rely on this, for example to calculate a delta between samples, would
need updating to take this into account. Previously valid event configurations (system-wide, no-inherit and so on) where each stream id
is the identifier are unaffected.
Quoted 3 lines
I think you meant
PERF_SAMPLE_IDnotPERF_SAMPLE_STREAM_ID. IIUC the stream id is already unique.
Yes my mistake.
Quoted 54 lines
Thanks,
NamhyungQuoted 51 lines
Changes since v3:
- Cleaned up perf test data changes incorrectly included into this series from elsewhere.Changes since v2:
- Rebase on v6.8
- Respond to James Clarke's feedback; fixup some typos and move
some
repeated checks into a helper macro.
- Cleaned up checkpatch lints.
- Updated perf test; fixed evsel handling so that existing tests
pass
and added new tests to cover the new behaviour.Changes since v1:
- Rebase on v6.8-rc1
- Fixed value written into sample after child exists.
- Modified handling of switch-out so that context with these
events
take the slow path, so that the per-event/per-thread PMU state
is
correctly switched.
- Modified perf tools to support this mode of operation.Ben Gainey (4):
perf: SupportPERF_SAMPLE_READwithinherit_stattools/perf: Track whereperf_sample_idsneed per-thread periodstools/perf: Correctly calculate sample period for inherited
SAMPLE_READvalues
tools/perf: Allow inherit +inherit_stat+PERF_SAMPLE_READwhen
opening eventsinclude/linux/perf_event.h | 1 + kernel/events/core.c | 62 ++++++++++---- tools/lib/perf/evlist.c | 1 + tools/lib/perf/evsel.c | 48 +++++++++++ tools/lib/perf/include/internal/evsel.h | 55 ++++++++++++- .../test-record-group-sampling-inherit-stat | 62 ++++++++++++++ tools/perf/util/evsel.c | 82 ++++++++++++++++++- tools/perf/util/evsel.h | 1 + tools/perf/util/session.c | 11 ++- 9 files changed, 301 insertions(+), 22 deletions(-) create mode 100644 tools/perf/tests/attr/test-record-group-sampling-inherit-stat
--
2.44.0
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Namhyung Kim
maintainerHello,
Quoted 54 lines · Ben Gainey
Quoted 24 lines · Namhyung Kim
Quoted 22 lines · Ben Gainey
This change allows events to use
PERF_SAMPLEREAD with inherit so long
as bothinherit_statandPERF_SAMPLE_TIDare set.Currently it is not possible to use
PERF_SAMPLE_READwith inherit. This
restriction assumes the user is interested in collecting aggregate statistics as perperf stat. It prevents a user from collecting per-thread samples using counter groups from a multi-threaded or multi-process application, as withperf record -e '{....}:S'. Instead
users must use system-wide mode, or forgo the ability to sample counter
groups. System-wide mode is often problematic as it requires specific
permissions (noCAP_PERFMON/ root access), or may lead to capture of
significant amounts of extra data from other processes running on the
system.Quoted 7 lines
Quoted 2 lines
Perf already supports the ability to collect per-thread counts with
inheritvia theinherit_statflag. This patch changesI'm not sure about this part. IIUC inherit and
inherit_statis not for
per-thread counts, it only supports per-process (including children) events.Hi Namhyung
Thanks for the comments...
I don't think this is correct, if you compare the behaviour of
perf record --no-inherit ... <some-forking-processes> perf script -F pid,tid | sort -u
and
perf record --no-inherit ... <some-multithreaded-processes> perf script -F pid,tid | sort -uvs
perf record ... <some-forking-processes>
perf script -F pid,tid | sort -u
and
perf record .. <some-multithreaded-processes>
perf script -F pid,tid | sort -u
Quoted 3 lines
The behaviour is consistent with the fact that no-inherit only records the primary thread of the primary process, whereas in the inherit case any child tasks (either threads or forked processes) is recorded.
Right, I was talking about the counting behavior not sampling as inherit_stat is only for the counting. I think it'd return an error if event attr has both sample_freq and inherit_stat.
Quoted 12 lines
Quoted 10 lines
Quoted 7 lines
perf_event_allocrelaxing the restriction to combineinheritwithPERF_SAMPLE_READso that the combination will be allowed so long asinherit_statandPERF_SAMPLE_TIDare enabled.Anyway, does it really need '
inherit_stat'? I think it's only for counting use cases (e.g. 'perf stat') not for sampling.
Quoted 5 lines
I would be very happy to remove the
inherit_statrequirement. When I first came to this it seemed like the logic was all there ininherit_statalready, but now that I have to take a different path inperf_event_context_sched_outI suspect it should be trivial to remove theinherit_statrequirement.
ok.
Quoted 23 lines
Quoted 3 lines
Also technically, it can have
PERF_SAMPLE_STREAM_IDinstead ofPERF_SAMPLE_TIDto distinguish the counter values.It looks like you are correct, but the ID given in the
read_formatpart ofPERF_SAMPLE_RECORDis the ID rather thanSTREAM_ID. (I had incorrectly thought/stated it was the latter). Hence when processing theread_formatvalues in the sample record, we either need to use the TID to uniquely identify the source, or we would need to modify theread_formatto (additionally) include theSTREAM_ID.* The current approach in tools uses the ID+TID, which puts more
complexity in the tools but means there isn't an extra field in theread_formatdata (for each value).
* Alternatively I could introduce aPERF_FORMAT_STREAM_ID; I would
expect that the user/tool would need to specify PERF_FORMAT_ID|PERF_FORMAT_STREAM_ID as they would need to use the ID to lookup the correctperf_event_attr, but could use theSTREAM_IDto uniquely identify the child event. This approach would add an extra u64 per value in theread_formatdata but is possibly simpler/safer for tools?
Quoted 1 line
Any preferences?
I think it's better to use TID + ID. IIUC there's no way to track STREAM_ID for new children other than getting it from sample. As sample has TID already it'd be meaningless using STREAM_ID to distinguish events.
Thanks,
Namhyung
Quoted 136 lines · Namhyung Kim
Hello,
Quoted 64 lines · Ben Gainey
Quoted 35 lines · Namhyung Kim
Quoted 29 lines · Ben Gainey
This change allows events to use
PERF_SAMPLEREAD with inherit so
long
as bothinherit_statandPERF_SAMPLE_TIDare set.Currently it is not possible to use
PERF_SAMPLE_READwith inherit.
This
restriction assumes the user is interested in collecting aggregate
statistics as perperf stat. It prevents a user from collecting
per-thread samples using counter groups from a multi-threaded or
multi-process application, as withperf record -e '{....}:S'. Instead
users must use system-wide mode, or forgo the ability to sample counter
groups. System-wide mode is often problematic as it requires specific
permissions (noCAP_PERFMON/ root access), or may lead to capture
of
significant amounts of extra data from other processes running on
the
system.Quoted 3 lines
Perf already supports the ability to collect per-thread counts with
inheritvia theinherit_statflag. This patch changesQuoted 6 lines
I'm not sure about this part. IIUC inherit and
inherit_statis not
for
per-thread counts, it only supports per-process (including children)
events.Hi Namhyung
Thanks for the comments...
I don't think this is correct, if you compare the behaviour of
perf record --no-inherit ... <some-forking-processes> perf script -F pid,tid | sort -u
and
perf record --no-inherit ... <some-multithreaded-processes> perf script -F pid,tid | sort -uvs
perf record ... <some-forking-processes>
perf script -F pid,tid | sort -u
and
perf record .. <some-multithreaded-processes>
perf script -F pid,tid | sort -uQuoted 5 lines
The behaviour is consistent with the fact that no-inherit only records
the primary thread of the primary process, whereas in the inherit case
any child tasks (either threads or forked processes) is recorded.Right, I was talking about the counting behavior not sampling as
inherit_statis only for the counting. I think it'd return an error
if event attr has bothsample_freqandinherit_stat.Quoted 15 lines
Quoted 10 lines
Quoted 9 lines
perf_event_allocrelaxing the restriction to combineinherit
withPERF_SAMPLE_READso that the combination will be allowed so long
asinherit_statandPERF_SAMPLE_TIDare enabled.Quoted 3 lines
Anyway, does it really need '
inherit_stat'? I think it's only for
counting use cases (e.g. 'perf stat') not for sampling.Quoted 8 lines
I would be very happy to remove the
inherit_statrequirement. When I
first came to this it seemed like the logic was all there ininherit_statalready, but now that I have to take a different path inperf_event_context_sched_outI suspect it should be trivial to remove
theinherit_statrequirement.ok.
Quoted 29 lines
Quoted 3 lines
Also technically, it can have
PERF_SAMPLE_STREAM_IDinstead ofPERF_SAMPLE_TIDto distinguish the counter values.It looks like you are correct, but the ID given in the
read_formatpart
ofPERF_SAMPLE_RECORDis the ID rather thanSTREAM_ID. (I had incorrectly thought/stated it was the latter). Hence when processing
theread_formatvalues in the sample record, we either need to use the
TID to uniquely identify the source, or we would need to modify theread_formatto (additionally) include theSTREAM_ID.* The current approach in tools uses the ID+TID, which puts more
complexity in the tools but means there isn't an extra field in theread_formatdata (for each value).
* Alternatively I could introduce aPERF_FORMAT_STREAM_ID; I would
expect that the user/tool would need to specify PERF_FORMAT_ID|PERF_FORMAT_STREAM_ID as they would need to use the ID
to lookup the correctperf_event_attr, but could use theSTREAM_IDto
uniquely identify the child event. This approach would add an extra u64
per value in theread_formatdata but is possibly simpler/safer for tools?
Quoted 6 lines
Quoted 1 line
Any preferences?
I think it's better to use TID + ID. IIUC there's no way to track
STREAM_IDfor new children other than getting it from sample. As sample has TID already it'd be meaningless usingSTREAM_IDto distinguish events.
So i did implement a prototype version of this where:
PERF_FORMAT_STREAM_ID is added, and perf record is modified to request that in addition to PERF_FORMAT_ID.
The PERF_FORMAT_ID is necessary to identify which perf_event_attr/evsel the counter represents, and the PERF_FORMAT_STREAM_ID uniquely identifies the child thread. It requires almost all the same plumbing as the previous implementation... I just modified my perf_sample_id__get_period_storage in patch 3/4 to take the u64 stream id instead of the u32 tid. I expect there could be some cleanup there, but I guess it highlights the fact that they amount to the same outcome.
I'll park that and push out a new patch set with just the inherit requirement.
Thanks
Ben
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.