Add table of BPF program types to docs
Subsystems (3)
v3 cover Add table of BPF program types to docs
Extend the libbpf documentation with a table of program types, attach points and ELF section names. The table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.
Patch 1 adds subdir support to Documentation/Makefile and changes userspace-api/media to use this instead of being a special case.
Patch 2 adds the program_types documentation with a new makefile in the libbpf doc directory to generate program_types.csv
I plan to look at adding info about the format of section "extras" for each program type as a follow-on.
v2 -> v3:
Put program_types after API docs in TOC as suggested by Andrii Nakryiko Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko Include sample of program_types.csv as suggested by Andrii Nakryiko
v1 -> v2:
Automate the generation of program_types.csv as suggested by Andrii Nakryiko.
Donald Hunter (2):
Add subdir support to Documentation makefile
Add table of BPF program types to libbpf docs
Documentation/Makefile | 16 ++++++- Documentation/bpf/libbpf/Makefile | 49 ++++++++++++++++++++++ Documentation/bpf/libbpf/index.rst | 3 ++ Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++ Documentation/bpf/programs.rst | 3 ++ Documentation/userspace-api/media/Makefile | 2 + 6 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 Documentation/bpf/libbpf/Makefile create mode 100644 Documentation/bpf/libbpf/program_types.rst
Signature · 1 line
2.35.1
PATCH v3 1/2 Add subdir support to Documentation makefile
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
--- Documentation/Makefile | 15 +++++++++++++-- Documentation/userspace-api/media/Makefile | 2 ++ 2 files changed, 15 insertions(+), 2 deletions(-)
Documentation/Makefile +13 -2
@@ -65,6 +65,12 @@ I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . # commands; the 'cmd' from scripts/Kbuild.include is not *loopable* loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit; +BUILD_SUBDIRS = \ + Documentation/userspace-api/media + +quiet_cmd_build_subdir = SUBDIR $2 + cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3 + # $2 sphinx builder e.g. "html" # $3 name of the build subfolder / e.g. "userspace-api/media", used as: # * dest folder relative to $(BUILDDIR) and
@@ -74,7 +80,7 @@ loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit; # e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4) - cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media $2 && \ + cmd_sphinx = \ PYTHONDONTWRITEBYTECODE=1 \ BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \ $(PYTHON3) $(srctree)/scripts/jobserver-exec \
@@ -93,6 +99,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4) htmldocs: @$(srctree)/scripts/sphinx-pre-install --version-check + @+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),html)) @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var))) linkcheckdocs:
@@ -100,6 +107,7 @@ linkcheckdocs: latexdocs: @$(srctree)/scripts/sphinx-pre-install --version-check + @+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex)) @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var))) ifeq ($(HAVE_PDFLATEX),0)
@@ -112,6 +120,7 @@ else # HAVE_PDFLATEX pdfdocs: latexdocs @$(srctree)/scripts/sphinx-pre-install --version-check + @+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex)) $(foreach var,$(SPHINXDIRS), \ $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \ mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -122,10 +131,12 @@ endif # HAVE_PDFLATEX epubdocs: @$(srctree)/scripts/sphinx-pre-install --version-check + @+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),epub)) @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var))) xmldocs: @$(srctree)/scripts/sphinx-pre-install --version-check + @+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),xml)) @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var))) endif # HAVE_SPHINX
@@ -138,7 +149,7 @@ refcheckdocs: cleandocs: $(Q)rm -rf $(BUILDDIR) - $(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media clean + @+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),clean)) dochelp: @echo ' Linux kernel internal documentation in different formats from ReST:'
Documentation/userspace-api/media/Makefile +2 -0
@@ -50,6 +50,8 @@ $(BUILDDIR)/lirc.h.rst: ${UAPI}/lirc.h ${PARSER} $(SRC_DIR)/lirc.h.rst.exception .PHONY: all html epub xml latex all: $(IMGDOT) $(BUILDDIR) ${TARGETS} + @: + html: all epub: all xml: all
Signature · 1 line
2.35.1
PATCH v3 2/2 Add table of BPF program types to libbpf docs
In reply to Donald HunterExtend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.
--- Documentation/Makefile | 3 +- Documentation/bpf/libbpf/Makefile | 49 ++++++++++++++++++++++ Documentation/bpf/libbpf/index.rst | 3 ++ Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++ Documentation/bpf/programs.rst | 3 ++ 5 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 Documentation/bpf/libbpf/Makefile create mode 100644 Documentation/bpf/libbpf/program_types.rst
Documentation/Makefile +2 -1
@@ -66,7 +66,8 @@ I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit; BUILD_SUBDIRS = \ - Documentation/userspace-api/media + Documentation/userspace-api/media \ + Documentation/bpf/libbpf quiet_cmd_build_subdir = SUBDIR $2 cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
Documentation/bpf/libbpf/Makefile new file +49 -0
49 lines not on this page · raw message
Documentation/bpf/libbpf/index.rst +3 -0
@@ -1,5 +1,7 @@ .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +.. _libbpf: + libbpf ======
@@ -7,6 +9,7 @@ libbpf :maxdepth: 1 API Documentation <https://libbpf.readthedocs.io/en/latest/api.html> + program_types libbpf_naming_convention libbpf_build
Documentation/bpf/libbpf/program_types.rst new file +32 -0
@@ -0,0 +1,32 @@ +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +.. _program_types_and_elf: + +Program Types and ELF Sections +============================== + +The table below lists the program types, their attach types where relevant and the ELF section +names supported by libbpf for them. The ELF section names follow these rules: + +- ``type`` is an exact match, e.g. ``SEC("socket")`` +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")`` + with a ‘``/``’ separator between ``type`` and ``extras``. + +When ``extras`` are specified, they provide details of how to auto-attach the BPF program. +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")`` +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes. + +.. + program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this: + Program Type,Attach Type,ELF Section Name,Sleepable + ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``, + ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes + +.. csv-table:: Program Types and Their ELF Section Names + :file: ../../output/program_types.csv + :widths: 40 30 20 10 + :header-rows: 1
Documentation/bpf/programs.rst +3 -0
@@ -7,3 +7,6 @@ Program Types :glob: prog_* + +For a list of all program types, see :ref:`program_types_and_elf` in +the :ref:`libbpf` documentation.
Signature · 1 line
2.35.1
Daniel Borkmann
maintainerRe: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile
In reply to Donald HunterQuoted 4 lines · Donald Hunter
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
Signed-off-by Donald Hunter <[email protected]>
Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.
Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.
Thanks,
Daniel
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
In reply to Donald HunterQuoted 4 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.
Oh, would be nice to turn all of these into proper but simple Sphinx extensions that take the deps into account in the Sphinx build process. And, of course, would be pure python instead of a combo of Make, shell, and awk.
That's one of the reasons we chose Sphinx originally, to be able to write Sphinx extensions and avoid complicated pipelines.
BR,
Jani.
Quoted 150 lines
Signed-off-by Donald Hunter <[email protected]>--- Documentation/Makefile | 3 +- Documentation/bpf/libbpf/Makefile | 49 ++++++++++++++++++++++ Documentation/bpf/libbpf/index.rst | 3 ++ Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++ Documentation/bpf/programs.rst | 3 ++ 5 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 Documentation/bpf/libbpf/Makefile create mode 100644 Documentation/bpf/libbpf/program_types.rstDocumentation/Makefile +2
-1@@ -66,7 +66,8 @@ I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit; BUILD_SUBDIRS = \ - Documentation/userspace-api/media + Documentation/userspace-api/media \ + Documentation/bpf/libbpf quiet_cmd_build_subdir = SUBDIR $2 cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3Documentation/bpf/libbpf/Makefile new file +49
-0@@ -0,0 +1,49 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Rules to convert BPF program types in tools/lib/bpf/libbpf.c +# into a .csv file + +FILES = program_types.csv + +TARGETS := $(addprefix $(BUILDDIR)/, $(FILES)) + +# Extract program types and properties from the section definitions in libbpf.c such as +# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv +# +# Here is a sample of the generated output that includes .rst formatting: +# +# Program Type,Attach Type,ELF Section Name,Sleepable +# ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``, +# ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``, +# ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``, +# ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``, +# ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``, +# ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes + +$(BUILDDIR)/program_types.csv: $(srctree)/tools/lib/bpf/libbpf.c + $(Q)awk -F'[",[:space:]]+' \ + 'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \ + /SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \ + type = "``BPF_PROG_TYPE_" $$4 "``"; \ + attach = index($$5, "0") ? "" : "``" $$5 "``"; \ + section = "``" $$3 "``"; \ + sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \ + print type "," attach "," section "," sleepable }' \ + $< > $@ + +.PHONY: all html epub xml latex linkcheck clean + +all: $(BUILDDIR) ${TARGETS} + @: + +html: all +epub: all +xml: all +latex: all +linkcheck: + +clean: + -$(Q)rm -f ${TARGETS} 2>/dev/null + +$(BUILDDIR): + $(Q)mkdir -p $@Documentation/bpf/libbpf/index.rst +3
-0@@ -1,5 +1,7 @@ .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +.. _libbpf: + libbpf ======@@ -7,6 +9,7 @@ libbpf :maxdepth: 1 API Documentation <https://libbpf.readthedocs.io/en/latest/api.html> + program_types libbpf_naming_convention libbpf_buildDocumentation/bpf/libbpf/program_types.rst new file +32
-0@@ -0,0 +1,32 @@ +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +.. _program_types_and_elf: + +Program Types and ELF Sections +============================== + +The table below lists the program types, their attach types where relevant and the ELF section +names supported by libbpf for them. The ELF section names follow these rules: + +- ``type`` is an exact match, e.g. ``SEC("socket")`` +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")`` + with a ‘``/``’ separator between ``type`` and ``extras``. + +When ``extras`` are specified, they provide details of how to auto-attach the BPF program. +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")`` +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes. + +.. + program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this: + Program Type,Attach Type,ELF Section Name,Sleepable + ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``, + ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes + +.. csv-table:: Program Types and Their ELF Section Names + :file: ../../output/program_types.csv + :widths: 40 30 20 10 + :header-rows: 1Documentation/bpf/programs.rst +3
-0@@ -7,3 +7,6 @@ Program Types :glob: prog_* + +For a list of all program types, see :ref:`program_types_and_elf` in +the :ref:`libbpf` documentation.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
In reply to Donald HunterQuoted 7 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.Signed-off-by Donald Hunter <[email protected]>---
[...]
Quoted 16 lines
Documentation/bpf/libbpf/program_types.rst new file +10
-0@@ -0,0 +1,32 @@ +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +.. _program_types_and_elf: + +Program Types and ELF Sections +============================== + +The table below lists the program types, their attach types where relevant and the ELF section +names supported by libbpf for them. The ELF section names follow these rules: +
Documentation/bpf/libbpf/program_types.rst line 20
Quoted 10 lines
Documentation/bpf/libbpf/program_types.rst new file +10
-0+- ``type`` is an exact match, e.g. ``SEC("socket")`` +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")`` + with a ‘``/``’ separator between ``type`` and ``extras``. + +When ``extras`` are specified, they provide details of how to auto-attach the BPF program. +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")`` +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes. + +.. + program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
^
Typo: s/fomatted/formatted
Documentation/bpf/libbpf/program_types.rst line 32
Quoted 12 lines
Documentation/bpf/libbpf/program_types.rst new file +12
-0+ Program Type,Attach Type,ELF Section Name,Sleepable + ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``, + ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes + +.. csv-table:: Program Types and Their ELF Section Names + :file: ../../output/program_types.csv + :widths: 40 30 20 10 + :header-rows: 1
Nice, I didn't know this formatting trick :-)
Appreciate you are working on this :-)
--Jesper
Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile
In reply to Daniel BorkmannQuoted 5 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
Signed-off-by Donald Hunter <[email protected]>
Quoted 7 lines
Jonathan, given this touches
Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.Maybe one small request, would be nice to build
Documentation/bpf/libbpf/also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.
Sure, I can look at what is needed for the BPF CI run.
Quoted 2 lines
Thanks,
Daniel
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
In reply to Jani NikulaQuoted 6 lines · Jani Nikula
Quoted 4 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.
Quoted 7 lines
Oh, would be nice to turn all of these into proper but simple Sphinx extensions that take the deps into account in the Sphinx build process. And, of course, would be pure python instead of a combo of Make, shell, and awk.
That's one of the reasons we chose Sphinx originally, to be able to write Sphinx extensions and avoid complicated pipelines.
I could look at this as a followup patch since I would need to learn how to write Sphinx extensions first. It seems that it will require a new reST directive, is that right?
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
Quoted 13 lines · Donald Hunter
Quoted 11 lines · Jani Nikula
Quoted 4 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.Oh, would be nice to turn all of these into proper but simple Sphinx extensions that take the deps into account in the Sphinx build process. And, of course, would be pure python instead of a combo of Make, shell, and awk.
Quoted 6 lines
Quoted 2 lines
That's one of the reasons we chose Sphinx originally, to be able to write Sphinx extensions and avoid complicated pipelines.
I could look at this as a followup patch since I would need to learn how to write Sphinx extensions first. It seems that it will require a new reST directive, is that right?
Correct. It's not very complicated. The extension that handles kernel-doc is under 200 lines, Documentation/sphinx/kerneldoc.py. And a lot of that is just translating directive options to kernel-doc the perl script options.
BR,
Jani.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
In reply to Donald HunterQuoted 131 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.Signed-off-by Donald Hunter <[email protected]>--- Documentation/Makefile | 3 +- Documentation/bpf/libbpf/Makefile | 49 ++++++++++++++++++++++ Documentation/bpf/libbpf/index.rst | 3 ++ Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++ Documentation/bpf/programs.rst | 3 ++ 5 files changed, 89 insertions(+), 1 deletion(-) create mode 100644 Documentation/bpf/libbpf/Makefile create mode 100644 Documentation/bpf/libbpf/program_types.rstDocumentation/Makefile +2
-1@@ -66,7 +66,8 @@ I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit; BUILD_SUBDIRS = \ - Documentation/userspace-api/media + Documentation/userspace-api/media \ + Documentation/bpf/libbpf quiet_cmd_build_subdir = SUBDIR $2 cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3Documentation/bpf/libbpf/Makefile new file +49
-0@@ -0,0 +1,49 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Rules to convert BPF program types in tools/lib/bpf/libbpf.c +# into a .csv file + +FILES = program_types.csv + +TARGETS := $(addprefix $(BUILDDIR)/, $(FILES)) + +# Extract program types and properties from the section definitions in libbpf.c such as +# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv +# +# Here is a sample of the generated output that includes .rst formatting: +# +# Program Type,Attach Type,ELF Section Name,Sleepable +# ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``, +# ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``, +# ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``, +# ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``, +# ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``, +# ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes + +$(BUILDDIR)/program_types.csv: $(srctree)/tools/lib/bpf/libbpf.c + $(Q)awk -F'[",[:space:]]+' \ + 'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \ + /SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \ + type = "``BPF_PROG_TYPE_" $$4 "``"; \ + attach = index($$5, "0") ? "" : "``" $$5 "``"; \ + section = "``" $$3 "``"; \ + sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \ + print type "," attach "," section "," sleepable }' \ + $< > $@ + +.PHONY: all html epub xml latex linkcheck clean + +all: $(BUILDDIR) ${TARGETS} + @: + +html: all +epub: all +xml: all +latex: all +linkcheck: + +clean: + -$(Q)rm -f ${TARGETS} 2>/dev/null + +$(BUILDDIR): + $(Q)mkdir -p $@Documentation/bpf/libbpf/index.rst +3
-0@@ -1,5 +1,7 @@ .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +.. _libbpf: + libbpf ======@@ -7,6 +9,7 @@ libbpf :maxdepth: 1 API Documentation <https://libbpf.readthedocs.io/en/latest/api.html> + program_types libbpf_naming_convention libbpf_buildDocumentation/bpf/libbpf/program_types.rst new file +20
-0@@ -0,0 +1,32 @@ +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +.. _program_types_and_elf: + +Program Types and ELF Sections +============================== + +The table below lists the program types, their attach types where relevant and the ELF section +names supported by libbpf for them. The ELF section names follow these rules: + +- ``type`` is an exact match, e.g. ``SEC("socket")`` +- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")`` + with a ‘``/``’ separator between ``type`` and ``extras``. + +When ``extras`` are specified, they provide details of how to auto-attach the BPF program. +The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")`` +for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes. + +.. + program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
Documentation/bpf/libbpf/program_types.rst line 30
Quoted 10 lines
Documentation/bpf/libbpf/program_types.rst new file +10
-0+ Program Type,Attach Type,ELF Section Name,Sleepable + ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``, + ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``, + ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``, + ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes + +.. csv-table:: Program Types and Their ELF Section Names + :file: ../../output/program_types.csv
Oh. You should probably test this with out-of-tree builds.
BR,
Jani.
Quoted 13 lines
Documentation/bpf/libbpf/program_types.rst new file +2
-0+ :widths: 40 30 20 10 + :header-rows: 1Documentation/bpf/programs.rst +3
-0@@ -7,3 +7,6 @@ Program Types :glob: prog_* + +For a list of all program types, see :ref:`program_types_and_elf` in +the :ref:`libbpf` documentation.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
Quoted 5 lines · Jani Nikula
Documentation/bpf/libbpf/program_types.rst line 30
Quoted 3 lines
Documentation/bpf/libbpf/program_types.rst new file +3
-0+ +.. csv-table:: Program Types and Their ELF Section Names + :file: ../../output/program_types.csvOh. You should probably test this with out-of-tree builds.
By out of tree, do you mean make htmldocs O=... or something else?
I have tested with O=... whih works for the kernel docs, but I think I will need to rework the location to support the libbpf doc build which has different directory layout.
Thanks,
Donald.
Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs
Quoted 9 lines · Donald Hunter
Quoted 5 lines · Jani Nikula
Documentation/bpf/libbpf/program_types.rst line 30
Quoted 3 lines
Documentation/bpf/libbpf/program_types.rst new file +3
-0+ +.. csv-table:: Program Types and Their ELF Section Names + :file: ../../output/program_types.csvOh. You should probably test this with out-of-tree builds.
By out of tree, do you mean make htmldocs O=... or something else?
Yeah, O= is what I mean.
BR,
Jani.
Quoted 7 lines
I have tested with O=... whih works for the kernel docs, but I think I will need to rework the location to support the libbpf doc build which has different directory layout.
Thanks,
Donald.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile
In reply to Donald HunterQuoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
Signed-off-by Donald Hunter <[email protected]>Jonathan, given this touches
Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.
Quoted 6 lines
Quoted 3 lines
Maybe one small request, would be nice to build
Documentation/bpf/libbpf/also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.Sure, I can look at what is needed for the BPF CI run.
Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?
Quoted 2 lines
Quoted 2 lines
Thanks,
Daniel
Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile
Quoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
Signed-off-by Donald Hunter <[email protected]>Jonathan, given this touches
Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Maybe one small request, would be nice to build
Documentation/bpf/libbpf/also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.Sure, I can look at what is needed for the BPF CI run.
Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?
It looks to me like it can be added to BPF CI if we change docs/conf.py to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can confirm whether this is the case.
Given that the tree layouts of the kernel and libbpf are very different I think we need to do one of:
1. Use a separate a .csv generator script with dedicated makefiles per project.
2. Move this entirely to libbpf and include a readthedocs URL in index.rst in the same way as currently done for the API docs.
I think my preference would be for 2.
Thanks
Donald.
Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile
Quoted 15 lines · Donald Hunter
Quoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
Signed-off-by Donald Hunter <[email protected]>Jonathan, given this touches
Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Maybe one small request, would be nice to build
Documentation/bpf/libbpf/also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.Sure, I can look at what is needed for the BPF CI run.
Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?
I think as long as all required packages are available in the CI distribution (which I believe is currently a Ubuntu image, but may in the future become Arch Linux) it should not be a problem to perform checking in CI. It seems as if generating the documentation may take a while, so we should likely try to have it run in a separate job. I can't tell what hidden dependencies there may be, though.
Quoted 3 lines
It looks to me like it can be added to BPF CI if we change
docs/conf.pyto call a new make target in docs/sphinx/Makefile. Hopefully Daniel can confirm whether this is the case.
I am not familiar with the documentation generation, but my quick search seems to suggest that this is done by a 3rd party service and is decoupled from BPF CI. Specifically, what you suggest may be reflected in the generated docs at
https://libbpf.readthedocs.io/, but I believe they are created from the libbpf GitHub repository, which is only infrequently synced from bpf tree sources. I didn't find any indication that CI triggers documentation creation, but it's possible I missed something.
[...]
Thanks,
Daniel
Quoted 39 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. The table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.Patch 1 adds subdir support to
Documentation/Makefileand changes userspace-api/media to use this instead of being a special case.Patch 2 adds the
program_typesdocumentation with a new makefile in the libbpf doc directory to generate program_types.csvI plan to look at adding info about the format of section "extras" for each program type as a follow-on.
v2 -> v3:
Putprogram_typesafter API docs in TOC as suggested by Andrii Nakryiko Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko Include sample of program_types.csv as suggested by Andrii Nakryikov1 -> v2:
Automate the generation of program_types.csv as suggested by Andrii Nakryiko.Donald Hunter (2):
Add subdir support to Documentation makefile
Add table of BPF program types to libbpf docsDocumentation/Makefile | 16 ++++++- Documentation/bpf/libbpf/Makefile | 49 ++++++++++++++++++++++ Documentation/bpf/libbpf/index.rst | 3 ++ Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++ Documentation/bpf/programs.rst | 3 ++ Documentation/userspace-api/media/Makefile | 2 + 6 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 Documentation/bpf/libbpf/Makefile create mode 100644 Documentation/bpf/libbpf/program_types.rst
Quoted 3 lines
--
2.35.1
This is marked as Changes Requested, so I presume there are some fixes/updates pending on top of v3? BPF CI should probably be done as a follow up, though.
But otherwise looks good to me and I appreciate the effort to improve libbpf's documentations.
Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile
In reply to Daniel MüllerQuoted 36 lines · Daniel Müller
Quoted 15 lines · Donald Hunter
Quoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter
Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.
Signed-off-by Donald Hunter <[email protected]>Jonathan, given this touches
Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Maybe one small request, would be nice to build
Documentation/bpf/libbpf/also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.Sure, I can look at what is needed for the BPF CI run.
Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?
I think as long as all required packages are available in the CI distribution (which I believe is currently a Ubuntu image, but may in the future become Arch Linux) it should not be a problem to perform checking in CI. It seems as if generating the documentation may take a while, so we should likely try to have it run in a separate job. I can't tell what hidden dependencies there may be, though.
Quoted 3 lines
It looks to me like it can be added to BPF CI if we change
docs/conf.pyto call a new make target in docs/sphinx/Makefile. Hopefully Daniel can confirm whether this is the case.
Quoted 7 lines
I am not familiar with the documentation generation, but my quick search seems to suggest that this is done by a 3rd party service and is decoupled from BPF CI. Specifically, what you suggest may be reflected in the generated docs at
https://libbpf.readthedocs.io/, but I believe they are created from the libbpf GitHub repository, which is only infrequently synced from bpf tree sources. I didn't find any indication that CI triggers documentation creation, but it's possible I missed something.
Apologies, I was referring to the libbpf GitHub repository. Anything committed to the kernel Documentation/bpf/libbpf risks breaking the documentation generation in the libbpf GitHub repository which, as you say, uses readthedocs.
Quoted 40 lines · Andrii Nakryiko
Quoted 39 lines · Donald Hunter
Extend the libbpf documentation with a table of program types, attach points and ELF section names. The table uses data from program_types.csv which is generated from
tools/lib/bpf/libbpf.cduring the documentation build.Patch 1 adds subdir support to
Documentation/Makefileand changes userspace-api/media to use this instead of being a special case.Patch 2 adds the
program_typesdocumentation with a new makefile in the libbpf doc directory to generate program_types.csvI plan to look at adding info about the format of section "extras" for each program type as a follow-on.
v2 -> v3:
Putprogram_typesafter API docs in TOC as suggested by Andrii Nakryiko Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko Include sample of program_types.csv as suggested by Andrii Nakryikov1 -> v2:
Automate the generation of program_types.csv as suggested by Andrii Nakryiko.Donald Hunter (2):
Add subdir support to Documentation makefile
Add table of BPF program types to libbpf docsDocumentation/Makefile | 16 ++++++- Documentation/bpf/libbpf/Makefile | 49 ++++++++++++++++++++++ Documentation/bpf/libbpf/index.rst | 3 ++ Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++ Documentation/bpf/programs.rst | 3 ++ Documentation/userspace-api/media/Makefile | 2 + 6 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 Documentation/bpf/libbpf/Makefile create mode 100644 Documentation/bpf/libbpf/program_types.rst
Quoted 7 lines
Quoted 3 lines
--
2.35.1This is marked as Changes Requested, so I presume there are some fixes/updates pending on top of v3? BPF CI should probably be done as a follow up, though.
Yes, there is a typo reported by Jesper Brouer that will require a v4.
Quoted 4 lines
But otherwise looks good to me and I appreciate the effort to improve libbpf's documentations.
Acked-by Andrii Nakryiko <[email protected]>
This patchset will break the documentation build in the libbpf GitHub repository when Documentation/bpf/libbpf gets synced to there.
At a minimum, I think I would need to refactor this patchset so that it is easier to integrate the .csv generation into the libbpf GitHub documentation build. But a simpler way forward might be to move this entirely to the libbpf Github repository and then just add a readthedocs URL to Documentation/bpf/libbpf/index.rst, the same as currently done for the API docs.