bpf: Add bpf_read_raw_record() helper
Subsystems (4)
PATCH 1/1 bpf: Add bpf_read_raw_record() helper
The helper is for BPF programs attached to perf_event in order to read event-specific raw data. I followed the convention of the bpf_read_branch_records() helper so that it can tell the size of record using BPF_F_GET_RAW_RECORD flag.
The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs a bpf_perf_event_data context so that's not an option IIUC.
include/uapi/linux/bpf.h | 23 ++++++++++++++++++++++ kernel/trace/bpf_trace.c | 41 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+)
include/uapi/linux/bpf.h +23 -0
@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value. +* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. +* +* **-EINVAL** if arguments invalid or **size** not a multiple +* of **sizeof**\ (u64\ ). +* +* **-ENOENT** if the event does not have raw records. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \
@@ -5566,6 +5583,7 @@ union bpf_attr {FN (tcp_raw_check_syncookie_ipv4), \FN (tcp_raw_check_syncookie_ipv6), \FN (ktime_get_tai_ns), \ +FN (read_raw_record), \/* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper
@@ -5749,6 +5767,11 @@ enum { BPF_F_EXCLUDE_INGRESS = (1ULL << 4), }; +/* BPF_FUNC_read_raw_record flags. */ +enum { +BPF_F_GET_RAW_RECORD_SIZE = (1ULL <<0 ), +}; +#define __bpf_md_ptr (type, name) \ union { \ type name; \
kernel/trace/bpf_trace.c +41 -0
59 lines not on this page · raw message
Signature · 1 line
2.37.2.609.g9ff673ca1a-goog
Quoted 18 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Quoted 6 lines
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.
Can we add a software event that generates raw data for testing?
Thanks,
Song
Quoted 122 lines
include/uapi/linux/bpf.h| 23 ++++++++++++++++++++++kernel/trace/bpf_trace.c| 41 ++++++++++++++++++++++++++++++++++++++++2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +23
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value. +* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. +* +* **-EINVAL** if arguments invalid or **size** not a multiple +* of **sizeof**\ (u64\ ). +* +* **-ENOENT** if the event does not have raw records. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -5566,6 +5583,7 @@ union bpf_attr {FN (tcp_raw_check_syncookie_ipv4), \FN (tcp_raw_check_syncookie_ipv6), \FN (ktime_get_tai_ns), \ +FN (read_raw_record), \/* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper @@ -5749,6 +5767,11 @@ enum { BPF_F_EXCLUDE_INGRESS = (1ULL << 4), }; +/* BPF_FUNC_read_raw_record flags. */ +enum { +BPF_F_GET_RAW_RECORD_SIZE = (1ULL <<0 ), +}; +#define __bpf_md_ptr (type, name) \ union { \ type name; \kernel/trace/bpf_trace.c +39
-0@@ -20,6 +20,7 @@#include <linux/fprobe.h> #include <linux/bsearch.h> #include <linux/sort.h> +#include <linux/perf_event.h> #include <net/bpf_sk_storage.h> @@ -1532,6 +1533,44 @@ static const struct bpf_func_proto bpf_read_branch_records_proto = { .arg4_type = ARG_ANYTHING, }; +BPF_CALL_4(bpf_read_raw_record, struct bpf_perf_event_data_kern *, ctx, + void *, buf, u32, size, u64, flags) +{ +struct perf_raw_record *raw = ctx->data->raw; +struct perf_raw_frag *frag; +u32 to_copy; + +if (unlikely (flags & ~BPF_F_GET_RAW_RECORD_SIZE )) +return -EINVAL ; + +if (unlikely (!raw)) +return -ENOENT ; + +if (flags &BPF_F_GET_RAW_RECORD_SIZE ) +return raw->size; + +if (!buf || (size %sizeof (u32) !=0 )) +return -EINVAL ; + + frag = &raw->frag; +WARN_ON_ONCE (!perf_raw_frag_last (frag)); + + to_copy =min_t (u32, frag->size, size); +memcpy (buf, frag->data, to_copy); + +return to_copy; +} + +static const struct bpf_func_proto bpf_read_raw_record_proto = { + .func = bpf_read_raw_record, + .gpl_only = true, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , + .arg2_type =ARG_PTR_TO_MEM_OR_NULL , + .arg3_type =ARG_CONST_SIZE_OR_ZERO , + .arg4_type =ARG_ANYTHING , +}; +static const struct
bpf_func_proto*pe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog){kernel/trace/bpf_trace.c +2
-0@@ -1548,6 +1587,8 @@ pe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_read_branch_records_proto;case BPF_FUNC_get_attach_cookie:return &bpf_get_attach_cookie_proto_pe; +case BPF_FUNC_read_raw_record: +return &bpf_read_raw_record_proto;default :return bpf_tracing_func_proto (func_id, prog); }Signature · 1 line
2.37.2.609.g9ff673ca1a-goog
Hi Song,
Quoted 19 lines · Song Liu
Quoted 19 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Quoted 8 lines
Quoted 6 lines
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.Can we add a software event that generates raw data for testing?
Ok, now I think that I can use a bpf-output sw event. It would need another BPF program to write data to the event and the test program can read it from BPF using this helper. :)
Thanks,
Namhyung
Quoted 29 lines · Namhyung Kim
Hi Song,
Quoted 26 lines · Song Liu
Quoted 25 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.
Quoted 8 lines
Quoted 1 line
Can we add a software event that generates raw data for testing?
Ok, now I think that I can use a bpf-output sw event. It would need another BPF program to write data to the event and the test program can read it from BPF using this helper. :)
Thanks,
Namhyung
Ah good idea. Feel free to carry my ACK to the v2 with the test.
Quoted 6 lines · John Fastabend
Quoted 3 lines · Namhyung Kim
Ok, now I think that I can use a bpf-output sw event. It would need another BPF program to write data to the event and the test program can read it from BPF using this helper. :)
Ah good idea. Feel free to carry my ACK to the v2 with the test.
Hmm.. it seems not to work because
1. bpf_output sw event doesn't have the overflow mechanism and it doesn't call the bpf program.
2. even if I added it, it couldn't run due to the recursion protection by bpf_prog_active.
Thanks,
Namhyung
Quoted 11 lines · Namhyung Kim
Quoted 5 lines · John Fastabend
Quoted 3 lines · Namhyung Kim
Ok, now I think that I can use a bpf-output sw event. It would need another BPF program to write data to the event and the test program can read it from BPF using this helper. :)
Quoted 7 lines
Quoted 1 line
Ah good idea. Feel free to carry my ACK to the v2 with the test.
Hmm.. it seems not to work because
1.bpf_outputsw event doesn't have the overflow mechanism and it doesn't call the bpf program.
2. even if I added it, it couldn't run due to the recursion protection bybpf_prog_active.
How about we enable some raw record for a software event? Something not controlled by BPF?
If this doesn't work, a self test that only runs on some hardware is also helpful.
Thanks,
Song
Quoted 12 lines · Song Liu
Quoted 12 lines · Namhyung Kim
Quoted 6 lines · John Fastabend
Quoted 3 lines · Namhyung Kim
Ok, now I think that I can use a bpf-output sw event. It would need another BPF program to write data to the event and the test program can read it from BPF using this helper. :)
Ah good idea. Feel free to carry my ACK to the v2 with the test.
Quoted 8 lines
Quoted 5 lines
Hmm.. it seems not to work because
1.bpf_outputsw event doesn't have the overflow mechanism and it doesn't call the bpf program.
2. even if I added it, it couldn't run due to the recursion protection bybpf_prog_active.How about we enable some raw record for a software event? Something not controlled by BPF?
Only for the test? It'd be nice if we could have meaningful data from software events but I don't have an idea what data it could carry on which event.
Peter, what do you think?
Quoted 3 lines
If this doesn't work, a self test that only runs on some hardware is also helpful.
Yep, makes sense.
Thanks,
Namhyung
Quoted 8 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Quoted 8 lines
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
Quoted 4 lines
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs a
bpf_perf_event_datacontext so that's not an option IIUC.
not a pmu expert but also no good ideas on my side.
...
Quoted 28 lines
kernel/trace/bpf_trace.c +27
-0+BPF_CALL_4(bpf_read_raw_record, struct bpf_perf_event_data_kern *, ctx, + void *, buf, u32, size, u64, flags) +{ +struct perf_raw_record *raw = ctx->data->raw; +struct perf_raw_frag *frag; +u32 to_copy; + +if (unlikely (flags & ~BPF_F_GET_RAW_RECORD_SIZE )) +return -EINVAL ; + +if (unlikely (!raw)) +return -ENOENT ; + +if (flags &BPF_F_GET_RAW_RECORD_SIZE ) +return raw->size; + +if (!buf || (size %sizeof (u32) !=0 )) +return -EINVAL ; + + frag = &raw->frag; +WARN_ON_ONCE (!perf_raw_frag_last (frag)); + + to_copy =min_t (u32, frag->size, size); +memcpy (buf, frag->data, to_copy); + +return to_copy; +}
kernel/trace/bpf_trace.c static const struct bpf_func_proto bpf_read_branch_records_proto = { · line 1572
Quoted 10 lines
kernel/trace/bpf_trace.c +10
-0+ +static const struct bpf_func_proto bpf_read_raw_record_proto = { + .func = bpf_read_raw_record, + .gpl_only = true, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , + .arg2_type =ARG_PTR_TO_MEM_OR_NULL , + .arg3_type =ARG_CONST_SIZE_OR_ZERO , + .arg4_type =ARG_ANYTHING , +};
Patch lgtm but curious why allow the ARG_PTR_TO_MEM_OR_NULL from API side instead of just ARG_PTR_TO_MEM? Maybe, just to match the existing perf_event_read()? I acked it as I think matching existing API is likely good enough reason.
Quoted 16 lines
kernel/trace/bpf_trace.c +3
-0+static const struct bpf_func_proto *pe_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog) {@@ -1548,6 +1587,8 @@ pe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_read_branch_records_proto;case BPF_FUNC_get_attach_cookie:return &bpf_get_attach_cookie_proto_pe; +case BPF_FUNC_read_raw_record: +return &bpf_read_raw_record_proto;default :return bpf_tracing_func_proto (func_id, prog); }Signature · 1 line
2.37.2.609.g9ff673ca1a-goog
Hello,
Quoted 16 lines · John Fastabend
Quoted 14 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Quoted 4 lines
Quoted 2 lines
Signed-off-by Namhyung Kim <[email protected]>---
Acked-by John Fastabend <[email protected]>
Thanks!
Quoted 45 lines
Quoted 5 lines
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs a
bpf_perf_event_datacontext so that's not an option IIUC.not a pmu expert but also no good ideas on my side.
...
Quoted 35 lines
kernel/trace/bpf_trace.c +34
-0+BPF_CALL_4(bpf_read_raw_record, struct bpf_perf_event_data_kern *, ctx, + void *, buf, u32, size, u64, flags) +{ +struct perf_raw_record *raw = ctx->data->raw; +struct perf_raw_frag *frag; +u32 to_copy; + +if (unlikely (flags & ~BPF_F_GET_RAW_RECORD_SIZE )) +return -EINVAL ; + +if (unlikely (!raw)) +return -ENOENT ; + +if (flags &BPF_F_GET_RAW_RECORD_SIZE ) +return raw->size; + +if (!buf || (size %sizeof (u32) !=0 )) +return -EINVAL ; + + frag = &raw->frag; +WARN_ON_ONCE (!perf_raw_frag_last (frag)); + + to_copy =min_t (u32, frag->size, size); +memcpy (buf, frag->data, to_copy); + +return to_copy; +} + +static const struct bpf_func_proto bpf_read_raw_record_proto = { + .func = bpf_read_raw_record, + .gpl_only = true, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_CTX , + .arg2_type =ARG_PTR_TO_MEM_OR_NULL ,
Quoted 8 lines
kernel/trace/bpf_trace.c static const struct bpf_func_proto bpf_read_branch_records_proto = { · line 1572
Quoted 3 lines
kernel/trace/bpf_trace.c +3
-0+ .arg3_type = ARG_CONST_SIZE_OR_ZERO, + .arg4_type = ARG_ANYTHING, +};Patch lgtm but curious why allow the
ARG_PTR_TO_MEM_OR_NULLfrom API side instead of justARG_PTR_TO_MEM? Maybe, just to match the existingperf_event_read()? I acked it as I think matching existing API is likely good enough reason.
It can query the size of raw record using BPF_F_GET_RAW_RECORD_SIZE. In that case it can pass NULL for the buffer (and 0 for the size).
Thanks,
Namhyung
Quoted 19 lines
Quoted 17 lines
kernel/trace/bpf_trace.c +3
-1+static const struct bpf_func_proto *pe_prog_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog) {@@ -1548,6 +1587,8 @@ pe_prog_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_read_branch_records_proto;case BPF_FUNC_get_attach_cookie:return &bpf_get_attach_cookie_proto_pe; +case BPF_FUNC_read_raw_record: +return &bpf_read_raw_record_proto;default :return bpf_tracing_func_proto (func_id, prog); } --2.37.2.609.g9ff673ca1a-goog
Quoted 36 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.include/uapi/linux/bpf.h | 23 ++++++++++++++++++++++ kernel/trace/bpf_trace.c | 41 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +2
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description
include/uapi/linux/bpf.h union bpf_attr { · line 5369
Quoted 10 lines
include/uapi/linux/bpf.h +10
-0+* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value. +* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL.
It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on BPF_F_GET_RAW_RECORD_SIZE.
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like bpf_perf_record_query(ctx, flags) where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (see bpf_ringbuf_query() for such approach). And then have separate helper to read data.
2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
And also, "read_raw_record" is way too generic. We have bpf_perf_prog_read_value(), let's use "bpf_perf_read_raw_record()" as a name. We should have called bpf_read_branch_records() as bpf_perf_read_branch_records(), probably, as well. But it's too late.
Quoted 15 lines
include/uapi/linux/bpf.h +6
-0+* +* **-EINVAL** if arguments invalid or **size** not a multiple +* of **sizeof**\ (u64\ ). +* +* **-ENOENT** if the event does not have raw records. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -5566,6 +5583,7 @@ union bpf_attr {FN (tcp_raw_check_syncookie_ipv4), \FN (tcp_raw_check_syncookie_ipv6), \FN (ktime_get_tai_ns), \ + FN(read_raw_record), \/* */
[...]
Quoted 47 lines · Andrii Nakryiko
Quoted 42 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.
include/uapi/linux/bpf.h| 23 ++++++++++++++++++++++kernel/trace/bpf_trace.c| 41 ++++++++++++++++++++++++++++++++++++++++2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +8
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value.
Quoted 8 lines
include/uapi/linux/bpf.h union bpf_attr { · line 5369
Quoted 4 lines
include/uapi/linux/bpf.h +4
-0+* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.
Yeah, I had the same thought when I first looked at it. But that's the exact syntax with bpf_read_branch_records(). Well, we still have time to fix the new helper..
Quoted 10 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.
Quoted 4 lines
2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
AFAICT, this is also confusing.
Maybe we should use two kfuncs for this?
Thanks,
Song
Quoted 24 lines
And also, "
read_raw_record" is way too generic. We havebpf_perf_prog_read_value(), let's use "bpf_perf_read_raw_record()" as a name. We should have calledbpf_read_branch_records()asbpf_perf_read_branch_records(), probably, as well. But it's too late.Quoted 15 lines
include/uapi/linux/bpf.h +6
-0+* +* **-EINVAL** if arguments invalid or **size** not a multiple +* of **sizeof**\ (u64\ ). +* +* **-ENOENT** if the event does not have raw records. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -5566,6 +5583,7 @@ union bpf_attr {FN (tcp_raw_check_syncookie_ipv4), \FN (tcp_raw_check_syncookie_ipv6), \FN (ktime_get_tai_ns), \ + FN(read_raw_record), \/* */ [...]
Quoted 71 lines · Song Liu
Quoted 48 lines · Andrii Nakryiko
Quoted 42 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.
include/uapi/linux/bpf.h| 23 ++++++++++++++++++++++kernel/trace/bpf_trace.c| 41 ++++++++++++++++++++++++++++++++++++++++2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +8
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value. Quoted 8 lines
include/uapi/linux/bpf.h union bpf_attr { · line 5369
Quoted 4 lines
include/uapi/linux/bpf.h +4
-0+* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.Yeah, I had the same thought when I first looked at it. But that's the exact syntax with
bpf_read_branch_records(). Well, we still have time to fix the new helper..Quoted 10 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.
Quoted 7 lines
Quoted 4 lines
2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
AFAICT, this is also confusing.
this is analogous to snprintf() behavior, so not that new and surprising when you think about it. But if query + read makes more sense, then it's fine by me
Quoted 30 lines
Maybe we should use two kfuncs for this?
Thanks,
SongQuoted 24 lines
And also, "
read_raw_record" is way too generic. We havebpf_perf_prog_read_value(), let's use "bpf_perf_read_raw_record()" as a name. We should have calledbpf_read_branch_records()asbpf_perf_read_branch_records(), probably, as well. But it's too late.Quoted 15 lines
include/uapi/linux/bpf.h +6
-0+* +* **-EINVAL** if arguments invalid or **size** not a multiple +* of **sizeof**\ (u64\ ). +* +* **-ENOENT** if the event does not have raw records. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -5566,6 +5583,7 @@ union bpf_attr {FN (tcp_raw_check_syncookie_ipv4), \FN (tcp_raw_check_syncookie_ipv6), \FN (ktime_get_tai_ns), \ + FN(read_raw_record), \/* */ [...]
Quoted 81 lines · Andrii Nakryiko
Quoted 76 lines · Song Liu
Quoted 48 lines · Andrii Nakryiko
Quoted 42 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.
include/uapi/linux/bpf.h| 23 ++++++++++++++++++++++kernel/trace/bpf_trace.c| 41 ++++++++++++++++++++++++++++++++++++++++2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +8
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value. Quoted 8 lines
include/uapi/linux/bpf.h union bpf_attr { · line 5369
Quoted 4 lines
include/uapi/linux/bpf.h +4
-0+* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.Yeah, I had the same thought when I first looked at it. But that's the exact syntax with
bpf_read_branch_records(). Well, we still have time to fix the new helper..Quoted 14 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
Quoted 6 lines
Quoted 2 lines
AFAICT, this is also confusing.
this is analogous to
snprintf()behavior, so not that new and surprising when you think about it. But if query + read makes more sense, then it's fine by me
Given the name discussion (the other email), I now like one API better.
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum perf_event_sample_format)? Something like:
long bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);
WDYT Namhyung?
Another idea is to add another parameter, so that we can pick which PERF_SAMPLE_* to output via bpf_perf_event_read_sample().
I think this will cover all cases with sample perf_event. Thoughts?
Thanks,
Song
Quoted 31 lines
Quoted 31 lines
Maybe we should use two kfuncs for this?
Thanks,
SongQuoted 24 lines
And also, "
read_raw_record" is way too generic. We havebpf_perf_prog_read_value(), let's use "bpf_perf_read_raw_record()" as a name. We should have calledbpf_read_branch_records()asbpf_perf_read_branch_records(), probably, as well. But it's too late.Quoted 15 lines
include/uapi/linux/bpf.h +6
-0+* +* **-EINVAL** if arguments invalid or **size** not a multiple +* of **sizeof**\ (u64\ ). +* +* **-ENOENT** if the event does not have raw records. */ #define __BPF_FUNC_MAPPER (FN ) \ FN(unspec), \@@ -5566,6 +5583,7 @@ union bpf_attr {FN (tcp_raw_check_syncookie_ipv4), \FN (tcp_raw_check_syncookie_ipv6), \FN (ktime_get_tai_ns), \ + FN(read_raw_record), \/* */ [...]
Quoted 57 lines · Song Liu
Quoted 48 lines · Andrii Nakryiko
Quoted 42 lines · Song Liu
Quoted 19 lines · Andrii Nakryiko
Quoted 5 lines · Namhyung Kim
+ * long
bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags)
+ * Description+ * For an eBPF program attached to a perf event, retrieve the + * raw record associated to *ctx* and store it in the buffer + * pointed by *buf* up to size *size* bytes.Quoted 7 lines
+ * Return
+ * On success, number of bytes written to *buf*. On error, a
+ * negative value.
+ *+ * The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to + * instead return the number of bytes required to store the raw + * record. If this flag is set, *buf* may be NULL.Quoted 3 lines
It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.Yeah, I had the same thought when I first looked at it. But that's the exact syntax with
bpf_read_branch_records(). Well, we still have time to fix the new helper..Quoted 14 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
Quoted 6 lines
Quoted 2 lines
AFAICT, this is also confusing.
this is analogous to
snprintf()behavior, so not that new and surprising when you think about it. But if query + read makes more sense, then it's fine by meGiven the name discussion (the other email), I now like one API better.
Quoted 7 lines
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
Quoted 5 lines
Another idea is to add another parameter, so that we can pick which PERF_SAMPLE_* to output via
bpf_perf_event_read_sample().I think this will cover all cases with sample
perf_event. Thoughts?
Yeah, I like this more and it looks easier to use.
Thanks,
Namhyung
Song Liu
maintainerQuoted 72 lines · Namhyung Kim
Quoted 57 lines · Song Liu
Quoted 48 lines · Andrii Nakryiko
Quoted 42 lines · Song Liu
Quoted 19 lines · Andrii Nakryiko
Quoted 5 lines · Namhyung Kim
+ * long
bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags)
+ * Description+ * For an eBPF program attached to a perf event, retrieve the + * raw record associated to *ctx* and store it in the buffer + * pointed by *buf* up to size *size* bytes.Quoted 7 lines
+ * Return
+ * On success, number of bytes written to *buf*. On error, a
+ * negative value.
+ *+ * The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to + * instead return the number of bytes required to store the raw + * record. If this flag is set, *buf* may be NULL.Quoted 3 lines
It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.Yeah, I had the same thought when I first looked at it. But that's the exact syntax with
bpf_read_branch_records(). Well, we still have time to fix the new helper..Quoted 14 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
Quoted 6 lines
Quoted 2 lines
AFAICT, this is also confusing.
this is analogous to
snprintf()behavior, so not that new and surprising when you think about it. But if query + read makes more sense, then it's fine by meGiven the name discussion (the other email), I now like one API better.
Quoted 7 lines
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Quoted 5 lines
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
The user should have access to the perf_event_attr used to create the event. This is also available in ctx->event->attr.
Would this work?
Thanks,
Song
Quoted 8 lines
Quoted 6 lines
Another idea is to add another parameter, so that we can pick which PERF_SAMPLE_* to output via
bpf_perf_event_read_sample().I think this will cover all cases with sample
perf_event. Thoughts?Yeah, I like this more and it looks easier to use.
Quoted 17 lines · Song Liu
Quoted 15 lines · Namhyung Kim
Quoted 7 lines · Song Liu
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Quoted 8 lines
Quoted 5 lines
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
The user should have access to the
perf_event_attrused to create the event. This is also available inctx->event->attr.
Do you mean from BPF? I'd like to have a generic BPF program that can handle various filtering according to the command line arguments. I'm not sure but it might do something differently for each event based on the attr settings.
Thanks,
Namhyung
Quoted 28 lines · Namhyung Kim
Quoted 23 lines · Song Liu
Quoted 20 lines · Namhyung Kim
Quoted 7 lines · Song Liu
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
Quoted 7 lines
Quoted 2 lines
The user should have access to the
perf_event_attrused to create the event. This is also available inctx->event->attr.Do you mean from BPF? I'd like to have a generic BPF program that can handle various filtering according to the command line arguments. I'm not sure but it might do something differently for each event based on the attr settings.
Yeah, we can access perf_event_attr from BPF program. Note that the ctx for perf_event bpf program is struct bpf_perf_event_data_kern:
SEC("perf_event")
int perf_e(struct bpf_perf_event_data_kern *ctx)
{
...
}struct bpf_perf_event_data_kern {
bpf_user_pt_regs_t *regs;
struct perf_sample_data *data;
struct perf_event *event;
};Alternatively, we can also have bpf user space configure the BPF program via a few knobs.
And actually, we can just read ctx->data and get the raw record, right..?
Thanks,
Song
Quoted 55 lines · Song Liu
Quoted 29 lines · Namhyung Kim
Quoted 23 lines · Song Liu
Quoted 20 lines · Namhyung Kim
Quoted 7 lines · Song Liu
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
Quoted 7 lines
Quoted 2 lines
The user should have access to the
perf_event_attrused to create the event. This is also available inctx->event->attr.Do you mean from BPF? I'd like to have a generic BPF program that can handle various filtering according to the command line arguments. I'm not sure but it might do something differently for each event based on the attr settings.
Yeah, we can access
perf_event_attrfrom BPF program. Note that the ctx forperf_eventbpf program is structbpf_perf_event_data_kern:
SEC("perf_event")
intperf_e(struct bpf_perf_event_data_kern *ctx){ ... }struct bpf_perf_event_data_kern { bpf_user_pt_regs_t *regs; struct perf_sample_data *data; struct perf_event *event; };
Quoted 5 lines
Alternatively, we can also have bpf user space configure the BPF program via a few knobs.
And actually, we can just read
ctx->dataand get the raw record, right..?
Played with this for a little bit. ctx->data appears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program before event->orig_overflow_handler. We can probably add a flag to specify we want to call orig_overflow_handler first.
Thanks,
Song
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.
I'm not sure. The sample_data should be provided by the caller of perf_event_overflow. So I guess the bpf program should see a valid ctx->data.
Also I want to control calling the orig_overflow_handler based on the return value of the BPF program. So calling the orig handler before BPF won't work for me. :)
Thanks,
Namhyung
Quoted 15 lines · Namhyung Kim
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.
Quoted 3 lines
I'm not sure. The
sample_datashould be provided by the caller ofperf_event_overflow. So I guess the bpf program should see a validctx->data.
Let's dig into this. Maybe we need some small changes in pe_prog_convert_ctx_access.
Quoted 3 lines
Also I want to control calling the
orig_overflow_handlerbased on the return value of the BPF program. So calling the orig handler before BPF won't work for me. :)
Interesting. Could you share more information about the use case?
Thanks,
Song
Quoted 16 lines · Song Liu
Quoted 16 lines · Namhyung Kim
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.
Quoted 6 lines
Quoted 3 lines
I'm not sure. The
sample_datashould be provided by the caller ofperf_event_overflow. So I guess the bpf program should see a validctx->data.Let's dig into this. Maybe we need some small changes in
pe_prog_convert_ctx_access.
Sure, can you explain the problem in detail and share your program?
Quoted 6 lines
Quoted 4 lines
Also I want to control calling the
orig_overflow_handlerbased on the return value of the BPF program. So calling the orig handler before BPF won't work for me. :)Interesting. Could you share more information about the use case?
Well.. it's nothing new. The bpf_overflow_handler calls the orig_overflow_handler (which writes the sample to the buffer) only if the BPF returns non zero. Then I can drop unnecessary samples based on the sample data by returning 0.
The possible use cases are
1. when you want to sample from specific code ranges only
2. when hardware sets specific bits in raw data
Thanks,
Namhyung
Quoted 21 lines · Namhyung Kim
Quoted 16 lines · Song Liu
Quoted 16 lines · Namhyung Kim
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I'm not sure. The
sample_datashould be provided by the caller ofperf_event_overflow. So I guess the bpf program should see a validctx->data.Let's dig into this. Maybe we need some small changes in
pe_prog_convert_ctx_access.Sure, can you explain the problem in detail and share your program?
I push the code to
https://git.kernel.org/pub/scm/linux/kernel/git/song/linux.git/log/?h=test-perf-event
The code is in tools/bpf/perf-test/.
The problem is we cannot get reliable print of data->cpu_entry in /sys/kernel/tracing/trace.
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Also I want to control calling the
orig_overflow_handlerbased on the return value of the BPF program. So calling the orig handler before BPF won't work for me. :)Interesting. Could you share more information about the use case?
Quoted 8 lines
Well.. it's nothing new. The
bpf_overflow_handlercalls theorig_overflow_handler(which writes the sample to the buffer) only if the BPF returns non zero. Then I can drop unnecessary samples based on the sample data by returning 0.The possible use cases are
1. when you want to sample from specific code ranges only
2. when hardware sets specific bits in raw data
I like this idea. We already using BPF in counting perf_event. Now it is time to use it in sampling perf_event. :)
Thanks,
Song
Quoted 31 lines · Song Liu
Quoted 22 lines · Namhyung Kim
Quoted 16 lines · Song Liu
Quoted 16 lines · Namhyung Kim
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I'm not sure. The
sample_datashould be provided by the caller ofperf_event_overflow. So I guess the bpf program should see a validctx->data.Let's dig into this. Maybe we need some small changes in
pe_prog_convert_ctx_access.Sure, can you explain the problem in detail and share your program?
Quoted 8 lines
I push the code to
https://git.kernel.org/pub/scm/linux/kernel/git/song/linux.git/log/?h=test-perf-event
The code is in
tools/bpf/perf-test/.The problem is we cannot get reliable print of
data->cpu_entryin/sys/kernel/tracing/trace.
Ah, right. I've realized that the sample data is passed before full initialized. Please see perf_sample_data_init(). The other members are initialized right before written to the ring buffer in the orig_overflow_handler (__perf_event_output).
That explains why pe_prog_convert_ctx_access() handles data and period specially. We need to handle it first.
Thanks,
Namhyung
Quoted 45 lines · Namhyung Kim
Quoted 31 lines · Song Liu
Quoted 22 lines · Namhyung Kim
Quoted 16 lines · Song Liu
Quoted 16 lines · Namhyung Kim
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I'm not sure. The
sample_datashould be provided by the caller ofperf_event_overflow. So I guess the bpf program should see a validctx->data.Let's dig into this. Maybe we need some small changes in
pe_prog_convert_ctx_access.Sure, can you explain the problem in detail and share your program?
Quoted 8 lines
I push the code to
https://git.kernel.org/pub/scm/linux/kernel/git/song/linux.git/log/?h=test-perf-event
The code is in
tools/bpf/perf-test/.The problem is we cannot get reliable print of
data->cpu_entryin/sys/kernel/tracing/trace.
Quoted 7 lines
Ah, right. I've realized that the sample data is passed before full initialized. Please see
perf_sample_data_init(). The other members are initialized right before written to the ring buffer in theorig_overflow_handler(__perf_event_output).That explains why
pe_prog_convert_ctx_access()handles data and period specially. We need to handle it first.
Thanks for confirming this. I guess we will need a helper (or kfunc) for the raw data.
Shall we make it more generic that we can get other PERF_SAMPLE_*?
Thanks,
Song
Quoted 51 lines · Song Liu
Quoted 51 lines · Namhyung Kim
Quoted 31 lines · Song Liu
Quoted 22 lines · Namhyung Kim
Quoted 16 lines · Song Liu
Quoted 16 lines · Namhyung Kim
Quoted 8 lines · Song Liu
Quoted 2 lines
And actually, we can just read
ctx->dataand get the raw record, right..?Played with this for a little bit.
ctx->dataappears to be not reliable sometimes. I guess (not 100% sure) this is because we call bpf program beforeevent->orig_overflow_handler. We can probably add a flag to specify we want to callorig_overflow_handlerfirst.Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I'm not sure. The
sample_datashould be provided by the caller ofperf_event_overflow. So I guess the bpf program should see a validctx->data.Let's dig into this. Maybe we need some small changes in
pe_prog_convert_ctx_access.Sure, can you explain the problem in detail and share your program?
Quoted 8 lines
I push the code to
https://git.kernel.org/pub/scm/linux/kernel/git/song/linux.git/log/?h=test-perf-event
The code is in
tools/bpf/perf-test/.The problem is we cannot get reliable print of
data->cpu_entryin/sys/kernel/tracing/trace.Ah, right. I've realized that the sample data is passed before full initialized. Please see
perf_sample_data_init(). The other members are initialized right before written to the ring buffer in theorig_overflow_handler(__perf_event_output).
Quoted 7 lines
Quoted 2 lines
That explains why
pe_prog_convert_ctx_access()handles data and period specially. We need to handle it first.Thanks for confirming this. I guess we will need a helper (or kfunc) for the raw data.
Shall we make it more generic that we can get other PERF_SAMPLE_*?
I don't think we can (or allow to) get all the sample data but some would be useful for filtering. Currently I'm only interested in the raw data, but ip and page size seem useful too.
So I think it'd be better to have a generic helper rather than a specific one. But it'd require some refactoring to get the data before calling BPF programs. Let me work on it first.
Thanks,
Namhyung
Quoted 46 lines · Song Liu
Quoted 29 lines · Namhyung Kim
Quoted 23 lines · Song Liu
Quoted 20 lines · Namhyung Kim
Quoted 7 lines · Song Liu
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
Quoted 7 lines
Quoted 2 lines
The user should have access to the
perf_event_attrused to create the event. This is also available inctx->event->attr.Do you mean from BPF? I'd like to have a generic BPF program that can handle various filtering according to the command line arguments. I'm not sure but it might do something differently for each event based on the attr settings.
Yeah, we can access
perf_event_attrfrom BPF program. Note that the ctx forperf_eventbpf program is structbpf_perf_event_data_kern:
SEC("perf_event")
intperf_e(struct bpf_perf_event_data_kern *ctx){ ... }
Quoted 5 lines
struct bpf_perf_event_data_kern {
bpf_user_pt_regs_t *regs;
struct perf_sample_data *data;
struct perf_event *event;
};I didn't know that it's allowed to access the kernel data directly. For some reason, I thought it should use fields in bpf_event_event_data only, like sample_period and addr. And the verifier will convert the access to them according to pe_prog_convert_ctx_access().
Quoted 6 lines
Alternatively, we can also have bpf user space configure the BPF program via a few knobs.
And actually, we can just read
ctx->dataand get the raw record, right..?
If it's possible, sure, it'd be more powerful.
Thanks,
Namhyung
Quoted 57 lines · Namhyung Kim
Quoted 46 lines · Song Liu
Quoted 29 lines · Namhyung Kim
Quoted 23 lines · Song Liu
Quoted 20 lines · Namhyung Kim
Quoted 7 lines · Song Liu
Actually, since we are on this, can we make it more generic, and handle all possible PERF_SAMPLE_* (in enum
perf_event_sample_format)? Something like:long
bpf_perf_event_read_sample(void *ctx, void *buf, u64 size, u64 flags);WDYT Namhyung?
Do you mean reading the whole sample data at once? Then it needs to parse the sample data format properly which is non trivial due to a number of variable length fields like callchains and branch stack, etc.
Also I'm afraid I might need event configuration info other than sample data like attr.type, attr.config, attr.sample_type and so on.
Hmm.. maybe we can add it to the ctx directly like ctx.attr_type?
Quoted 7 lines
Quoted 2 lines
The user should have access to the
perf_event_attrused to create the event. This is also available inctx->event->attr.Do you mean from BPF? I'd like to have a generic BPF program that can handle various filtering according to the command line arguments. I'm not sure but it might do something differently for each event based on the attr settings.
Yeah, we can access
perf_event_attrfrom BPF program. Note that the ctx forperf_eventbpf program is structbpf_perf_event_data_kern:
SEC("perf_event")
intperf_e(struct bpf_perf_event_data_kern *ctx){ ... }Quoted 5 lines
struct bpf_perf_event_data_kern { bpf_user_pt_regs_t *regs; struct perf_sample_data *data; struct perf_event *event; };
Quoted 4 lines
I didn't know that it's allowed to access the kernel data directly. For some reason, I thought it should use fields in
bpf_event_event_dataonly, likesample_periodand addr. And the verifier will convert the access to them according tope_prog_convert_ctx_access().
We can bypass pe_prog_convert_ctx_access() with something like:
struct perf_event *event;
u64 config;
bpf_probe_read_kernel(&event, sizeof(void *), &ctx->event);
bpf_probe_read_kernel(&config, sizeof(u64), &event->attr.config);Thanks,
Song
Hello,
Quoted 44 lines · Andrii Nakryiko
Quoted 42 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.include/uapi/linux/bpf.h | 23 ++++++++++++++++++++++ kernel/trace/bpf_trace.c | 41 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +8
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value.
Quoted 8 lines
include/uapi/linux/bpf.h union bpf_attr { · line 5369
Quoted 4 lines
include/uapi/linux/bpf.h +4
-0+* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.
Agreed.
Quoted 9 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.
I like this as I want to have more info for the perf event sample like instruction address or sample type. I know some of the info is available through the context but I think this is a better approach.
Quoted 11 lines
2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
And also, "
read_raw_record" is way too generic. We havebpf_perf_prog_read_value(), let's use "bpf_perf_read_raw_record()" as a name. We should have calledbpf_read_branch_records()asbpf_perf_read_branch_records(), probably, as well. But it's too late.
Yeah, what about this?
* bpf_perf_event_query(ctx, flag)
* bpf_perf_event_get(ctx, flag, buf, size)
Maybe we can use the same flag for both. Like BPF_PERF_RAW_RECORD can return the size (or -1 if not) on _query() and read the data on _get(). Or we can have a BPF_PERF_RAW_RECORD_SIZE only for _query().
It seems we don't need _get() for things like BPF_PERF_SAMPLE_IP. What do you think?
Thanks,
Namhyung
Quoted 88 lines · Namhyung Kim
Hello,
Quoted 44 lines · Andrii Nakryiko
Quoted 42 lines · Namhyung Kim
The helper is for BPF programs attached to
perf_eventin order to read event-specific raw data. I followed the convention of thebpf_read_branch_records()helper so that it can tell the size of record usingBPF_F_GET_RAW_RECORDflag.The use case is to filter perf event samples based on the HW provided data which have more detailed information about the sample.
Note that it only reads the first fragment of the raw record. But it seems mostly ok since all the existing PMU raw data have only single fragment and the multi-fragment records are only for BPF output attached to sockets. So unless it's used with such an extreme case, it'd work for most of tracing use cases.
Signed-off-by Namhyung Kim <[email protected]>---
I don't know how to test this. As the raw data is available on some hardware PMU only (e.g. AMD IBS). I tried a tracepoint event but it was rejected by the verifier. Actually it needs abpf_perf_event_datacontext so that's not an option IIUC.include/uapi/linux/bpf.h | 23 ++++++++++++++++++++++ kernel/trace/bpf_trace.c | 41 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+)include/uapi/linux/bpf.h +8
-0@@ -5355,6 +5355,23 @@ union bpf_attr {* Return * Current *ktime*. * +* long bpf_read_raw_record(struct bpf_perf_event_data *ctx, void *buf, u32 size, u64 flags) +* Description +* For an eBPF program attached to a perf event, retrieve the +* raw record associated to *ctx* and store it in the buffer +* pointed by *buf* up to size *size* bytes. +* Return +* On success, number of bytes written to *buf*. On error, a +* negative value. Quoted 8 lines
include/uapi/linux/bpf.h union bpf_attr { · line 5369
Quoted 4 lines
include/uapi/linux/bpf.h +4
-0+* +* The *flags* can be set to **BPF_F_GET_RAW_RECORD_SIZE** to +* instead return the number of bytes required to store the raw +* record. If this flag is set, *buf* may be NULL. It looks pretty ugly from a usability standpoint to have one helper doing completely different things and returning two different values based on
BPF_F_GET_RAW_RECORD_SIZE.Agreed.
Quoted 9 lines
I'm not sure what's best, but I have two alternative proposals:
1. Add two helpers: one to get perf record information (and size will be one of them). Something like
bpf_perf_record_query(ctx, flags)where you pass perf ctx and what kind of information you want to read (through flags), and u64 return result returns that (seebpf_ringbuf_query()for such approach). And then have separate helper to read data.I like this as I want to have more info for the perf event sample like instruction address or sample type. I know some of the info is available through the context but I think this is a better approach.
Quoted 11 lines
2. Keep one helper, but specify that it always returns record size, even if user specified smaller size to read. And then allow passing buf==NULL && size==0. So passing NULL, 0 -- you get record size. Passing non-NULL buf -- you read data.
And also, "
read_raw_record" is way too generic. We havebpf_perf_prog_read_value(), let's use "bpf_perf_read_raw_record()" as a name. We should have calledbpf_read_branch_records()asbpf_perf_read_branch_records(), probably, as well. But it's too late.Yeah, what about this?
Quoted 9 lines
*
bpf_perf_event_query(ctx, flag)
*bpf_perf_event_get(ctx, flag, buf, size)Maybe we can use the same flag for both. Like
BPF_PERF_RAW_RECORDcan return the size (or -1 if not) on_query()and read the data on_get(). Or we can have aBPF_PERF_RAW_RECORD_SIZEonly for_query().
It seems we don't need_get()for things likeBPF_PERF_SAMPLE_IP. What do you think?
probably separate flags makes more sense, because I can see how we can allow querying multiple up-to-u64-sized things (even sample_ip, for example), while reserve bpf_perf_event_get() to variable-size or big sized values.
Not super keen on "get", "read" or "load" is the verb we typically use for such operations, but we already have bpf_perf_event_read(). And bpf_perf_event_load() doesn't seem right either. Naming is hard. Maybe "fetch"? Or just "get". Don't know, but maybe someone has good ideas.
Quoted 2 lines
Thanks,
Namhyung