BPF program metadata
Subsystems (3)
As discussed in
https://docs.google.com/presentation/d/1A9Anx8JPHl_pK1aXy8hlxs3V5pkrKwHHtkPf_-HeYTc
during BPF office hours, we'd like to attach arbitrary auxiliary metadata to the program, for example, the build timestamp or the commit hash.
IIRC, the suggestion was to explore BTF and .BTF.ext section in particular.
We've spent some time looking at the BTF encoding and BTF.ext section and we don't see how we can put this data into .BTF.ext or even .BTF without any kernel changes.
The reasoning (at least how we see it):
* .BTF.ext is just a container with func_info/line_info/relocation_info and libbpf extracts the data form this section and passes it to sys_bpf(BPF_PROG_LOAD); the important note is that it doesn't pass the whole container to the kernel, but passes the data that's been extracted from the appropriate sections
* .BTF can be used for metadata, but it looks like we'd have to add another BTF_INFO_KIND() to make it a less messy (YiFei, feel free to correct me)
So the question is: are we missing something? Is there some way to add key=value metadata to BTF that doesn't involve a lot of kernel changes?
If the restrictions above are correct, should we go back to trying to put this metadata into .data section (or maybe even the new .metadata section)? The only missing piece of the puzzle in that case is the ability to extend BPF_PROG_LOAD with a way to say 'hold this map unconditionally'.
Quoted 22 lines · On 07/29, [email protected] wrote:
As discussed in
https://docs.google.com/presentation/d/1A9Anx8JPHl_pK1aXy8hlxs3V5pkrKwHHtkPf_-HeYTc
during BPF office hours, we'd like to attach arbitrary auxiliary metadata to the program, for example, the build timestamp or the commit hash.IIRC, the suggestion was to explore BTF and .BTF.ext section in particular.
We've spent some time looking at the BTF encoding and BTF.ext section and we don't see how we can put this data into .BTF.ext or even .BTF without any kernel changes.The reasoning (at least how we see it):
* .BTF.ext is just a container with func_info/line_info/relocation_info and libbpf extracts the data form this section and passes it tosys_bpf(BPF_PROG_LOAD); the important note is that it doesn't pass the whole container to the kernel, but passes the data that's been extracted from the appropriate sections
* .BTF can be used for metadata, but it looks like we'd have to add anotherBTF_INFO_KIND()to make it a less messy (YiFei, feel free to correct me)
Quoted 8 lines
So the question is: are we missing something? Is there some way to add key=value metadata to BTF that doesn't involve a lot of kernel changes?
If the restrictions above are correct, should we go back to trying to put this metadata into .data section (or maybe even the new .metadata section)? The only missing piece of the puzzle in that case is the ability to extend
BPF_PROG_LOADwith a way to say 'hold this map unconditionally'.
Should we have a short discussion about that this Thu during the office hours? I don't see this week's sheet page in there yet :-(
Quoted 27 lines · On Tue, Aug 4, 2020 at 12:42 PM <[email protected]> wrote:
Quoted 24 lines · On 07/29, [email protected] wrote:
As discussed in
https://docs.google.com/presentation/d/1A9Anx8JPHl_pK1aXy8hlxs3V5pkrKwHHtkPf_-HeYTc
during BPF office hours, we'd like to attach arbitrary auxiliary metadata to the program, for example, the build timestamp or the commit hash.IIRC, the suggestion was to explore BTF and .BTF.ext section in particular.
We've spent some time looking at the BTF encoding and BTF.ext section and we don't see how we can put this data into .BTF.ext or even .BTF without any kernel changes.The reasoning (at least how we see it):
* .BTF.ext is just a container with func_info/line_info/relocation_info and libbpf extracts the data form this section and passes it tosys_bpf(BPF_PROG_LOAD); the important note is that it doesn't pass the whole container to the kernel, but passes the data that's been extracted from the appropriate sections
* .BTF can be used for metadata, but it looks like we'd have to add anotherBTF_INFO_KIND()to make it a less messy (YiFei, feel free to correct me)So the question is: are we missing something? Is there some way to add key=value metadata to BTF that doesn't involve a lot of kernel changes?
Quoted 7 lines
Quoted 5 lines
If the restrictions above are correct, should we go back to trying to put this metadata into .data section (or maybe even the new .metadata section)? The only missing piece of the puzzle in that case is the ability to extend
BPF_PROG_LOADwith a way to say 'hold this map unconditionally'.Should we have a short discussion about that this Thu during the office hours?
Of course. That's what office hours are for.
Since google folks have trouble with zoom I've added google meets link to the spreadsheet. Let's try it tomorrow.
Quoted 39 lines · Alexei Starovoitov
Quoted 4 lines · On Tue, Aug 4, 2020 at 12:42 PM <[email protected]> wrote:
Quoted 2 lines · On 07/29, [email protected] wrote:
As discussed in
https://docs.google.com/presentation/d/1A9Anx8JPHl_pK1aXy8hlxs3V5pkrKwHHtkPf_-HeYTc
Quoted 2 lines
Quoted 2 lines
during BPF office hours, we'd like to attach arbitrary auxiliary metadata to the program, for example, the build timestamp or the
commit
Quoted 10 lines
Quoted 10 lines
hash.
IIRC, the suggestion was to explore BTF and .BTF.ext section in particular.
We've spent some time looking at the BTF encoding and BTF.ext section and we don't see how we can put this data into .BTF.ext or even .BTF without any kernel changes.The reasoning (at least how we see it):
* .BTF.ext is just a container withfunc_info/line_info/relocation_info
Quoted 2 lines
Quoted 2 lines
and libbpf extracts the data form this section and passes it to
sys_bpf(BPF_PROG_LOAD); the important note is that it doesn't passthe
Quoted 4 lines
Quoted 4 lines
whole container to the kernel, but passes the data that's been extracted from the appropriate sections
* .BTF can be used for metadata, but it looks like we'd have to add anotherBTF_INFO_KIND()to make it a less messy (YiFei, feel freeto
Quoted 4 lines
Quoted 4 lines
correct me)
So the question is: are we missing something? Is there some way to add key=value metadata to BTF that doesn't involve a lot of kernel
changes?
Quoted 6 lines
Quoted 6 lines
If the restrictions above are correct, should we go back to trying to put this metadata into .data section (or maybe even the new .metadata section)? The only missing piece of the puzzle in that case is the ability to extend
BPF_PROG_LOADwith a way to say 'hold this map unconditionally'.
Quoted 6 lines
Quoted 2 lines
Should we have a short discussion about that this Thu during the office hours?
Of course. That's what office hours are for.
Since google folks have trouble with zoom I've added google meets link to the spreadsheet. Let's try it tomorrow.
Ooooh, thank you for the Google Meets link :-)
BPF office hours via google meet. Was: BPF program metadata
Quoted 2 lines · On Wed, Aug 5, 2020 at 4:10 PM <[email protected]> wrote:
Quoted 2 lines
Since google folks have trouble with zoom I've added google meets link to the spreadsheet. Let's try it tomorrow.
Today we held BPF office hours via google meet and it worked well. So we will be using it for future meetings.
zoom link has been removed from the spreadsheet to avoid confusion.