Skip to contentSkip to conversation

Add table of BPF program types to docs

PATCH Superseded Donald Hunter 6 participants 18 messages Aug 29, 2022 – Sep 12, 2022

Subsystems (3)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover Add table of BPF program types to docs

Extend the libbpf documentation with a table of program types, attach points and ELF section names. The table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

Patch 1 adds subdir support to Documentation/Makefile and changes userspace-api/media to use this instead of being a special case.

Patch 2 adds the program_types documentation with a new makefile in the libbpf doc directory to generate program_types.csv

I plan to look at adding info about the format of section "extras" for each program type as a follow-on.

v2 -> v3:
Put program_types after API docs in TOC as suggested by Andrii Nakryiko Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko Include sample of program_types.csv as suggested by Andrii Nakryiko

v1 -> v2:
Automate the generation of program_types.csv as suggested by Andrii Nakryiko.

Donald Hunter (2):
Add subdir support to Documentation makefile
Add table of BPF program types to libbpf docs

 Documentation/Makefile                     | 16 ++++++-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 Documentation/userspace-api/media/Makefile |  2 +
 6 files changed, 103 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
Signature · 1 line
2.35.1
#1 lore raw

PATCH v3 1/2 Add subdir support to Documentation makefile

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

---
 Documentation/Makefile                     | 15 +++++++++++++--
 Documentation/userspace-api/media/Makefile |  2 ++
 2 files changed, 15 insertions(+), 2 deletions(-)
Documentation/Makefile +13 -2
index 64d44c1ecad3..8a63ef2dcd1c 100644
@@ -65,6 +65,12 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 # commands; the 'cmd' from scripts/Kbuild.include is not *loopable*
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
+BUILD_SUBDIRS = \
+	Documentation/userspace-api/media
+
+quiet_cmd_build_subdir = SUBDIR  $2
+      cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
+
 # $2 sphinx builder e.g. "html"
 # $3 name of the build subfolder / e.g. "userspace-api/media", used as:
 #    * dest folder relative to $(BUILDDIR) and
@@ -74,7 +80,7 @@ loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 #    e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media
 
 quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
-      cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media $2 && \
+      cmd_sphinx =  \
 	PYTHONDONTWRITEBYTECODE=1 \
 	BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
 	$(PYTHON3) $(srctree)/scripts/jobserver-exec \
@@ -93,6 +99,7 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
 
 htmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),html))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
 
 linkcheckdocs:
@@ -100,6 +107,7 @@ linkcheckdocs:
 
 latexdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
 
 ifeq ($(HAVE_PDFLATEX),0)
@@ -112,6 +120,7 @@ else # HAVE_PDFLATEX
 
 pdfdocs: latexdocs
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),latex))
 	$(foreach var,$(SPHINXDIRS), \
 	   $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit; \
 	   mkdir -p $(BUILDDIR)/$(var)/pdf; \
@@ -122,10 +131,12 @@ endif # HAVE_PDFLATEX
 
 epubdocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),epub))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
 
 xmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),xml))
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
 
 endif # HAVE_SPHINX
@@ -138,7 +149,7 @@ refcheckdocs:
 
 cleandocs:
 	$(Q)rm -rf $(BUILDDIR)
-	$(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/userspace-api/media clean
+	@+$(foreach var,$(BUILD_SUBDIRS),$(call loop_cmd,build_subdir,$(var),clean))
 
 dochelp:
 	@echo  ' Linux kernel internal documentation in different formats from ReST:'
Documentation/userspace-api/media/Makefile +2 -0
index 00922aa7efde..783c6f880b72 100644
@@ -50,6 +50,8 @@ $(BUILDDIR)/lirc.h.rst: ${UAPI}/lirc.h ${PARSER} $(SRC_DIR)/lirc.h.rst.exception
 .PHONY: all html epub xml latex
 
 all: $(IMGDOT) $(BUILDDIR) ${TARGETS}
+	@:
+
 html: all
 epub: all
 xml: all
Signature · 1 line
2.35.1
maintainer
#2 lore raw

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

Quoted 4 lines · Donald Hunter

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.

Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.

Thanks,
Daniel

#3 lore raw

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

Quoted 5 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

Quoted 7 lines

Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.

Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.

Sure, I can look at what is needed for the BPF CI run.

Quoted 2 lines

Thanks,
Daniel

#4 lore raw

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.

Quoted 6 lines
Quoted 3 lines

Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.

Sure, I can look at what is needed for the BPF CI run.

Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

Quoted 2 lines
Quoted 2 lines

Thanks,
Daniel

#5 lore raw

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

Quoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.

Sure, I can look at what is needed for the BPF CI run.

Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

It looks to me like it can be added to BPF CI if we change docs/conf.py to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can confirm whether this is the case.

Given that the tree layouts of the kernel and libbpf are very different I think we need to do one of:

1. Use a separate a .csv generator script with dedicated makefiles per project.

2. Move this entirely to libbpf and include a readthedocs URL in index.rst in the same way as currently done for the API docs.

I think my preference would be for 2.

Thanks
Donald.

#6 lore raw

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

Quoted 15 lines · Donald Hunter
Quoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.

Sure, I can look at what is needed for the BPF CI run.

Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

I think as long as all required packages are available in the CI distribution (which I believe is currently a Ubuntu image, but may in the future become Arch Linux) it should not be a problem to perform checking in CI. It seems as if generating the documentation may take a while, so we should likely try to have it run in a separate job. I can't tell what hidden dependencies there may be, though.

Quoted 3 lines

It looks to me like it can be added to BPF CI if we change docs/conf.py to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can confirm whether this is the case.

I am not familiar with the documentation generation, but my quick search seems to suggest that this is done by a 3rd party service and is decoupled from BPF CI. Specifically, what you suggest may be reflected in the generated docs at
https://libbpf.readthedocs.io/, but I believe they are created from the libbpf GitHub repository, which is only infrequently synced from bpf tree sources. I didn't find any indication that CI triggers documentation creation, but it's possible I missed something.

[...]

Thanks,
Daniel

#7 lore raw

Re: [PATCH bpf-next v3 1/2] Add subdir support to Documentation makefile

Quoted 36 lines · Daniel Müller
Quoted 15 lines · Donald Hunter
Quoted 13 lines · Andrii Nakryiko
Quoted 12 lines · Donald Hunter
Quoted 9 lines · Daniel Borkmann
Quoted 3 lines · Donald Hunter

Run make in list of subdirs to build generated sources and migrate userspace-api/media to use this instead of being a special case.

Jonathan, given this touches Documentation/Makefile, could you ACK if it looks good to you? Noticed both patches don't have doc: $subj prefix, but that's something we could fix up.

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Maybe one small request, would be nice to build Documentation/bpf/libbpf/ also with every BPF CI run to avoid breakage of program_types.csv. Donald could you check if feasible? Follow-up might be ok too, but up to Andrii.

Sure, I can look at what is needed for the BPF CI run.

Daniel (Mueller, not Borkmann), is this something that can be added to BPF CI?

I think as long as all required packages are available in the CI distribution (which I believe is currently a Ubuntu image, but may in the future become Arch Linux) it should not be a problem to perform checking in CI. It seems as if generating the documentation may take a while, so we should likely try to have it run in a separate job. I can't tell what hidden dependencies there may be, though.

Quoted 3 lines

It looks to me like it can be added to BPF CI if we change docs/conf.py to call a new make target in docs/sphinx/Makefile. Hopefully Daniel can confirm whether this is the case.

Quoted 7 lines

I am not familiar with the documentation generation, but my quick search seems to suggest that this is done by a 3rd party service and is decoupled from BPF CI. Specifically, what you suggest may be reflected in the generated docs at
https://libbpf.readthedocs.io/, but I believe they are created from the libbpf GitHub repository, which is only infrequently synced from bpf tree sources. I didn't find any indication that CI triggers documentation creation, but it's possible I missed something.

Apologies, I was referring to the libbpf GitHub repository. Anything committed to the kernel Documentation/bpf/libbpf risks breaking the documentation generation in the libbpf GitHub repository which, as you say, uses readthedocs.

#8 lore raw

PATCH v3 2/2 Add table of BPF program types to libbpf docs

In reply to Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

---
 Documentation/Makefile                     |  3 +-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 5 files changed, 89 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
Documentation/Makefile +2 -1
index 8a63ef2dcd1c..f007314770e1 100644
@@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
 BUILD_SUBDIRS = \
-	Documentation/userspace-api/media
+	Documentation/userspace-api/media \
+	Documentation/bpf/libbpf
 
 quiet_cmd_build_subdir = SUBDIR  $2
       cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
Documentation/bpf/libbpf/Makefile new file +49 -0

49 lines not on this page · raw message

Documentation/bpf/libbpf/index.rst +3 -0
index 3722537d1384..f9b3b252e28f 100644
@@ -1,5 +1,7 @@
 .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+.. _libbpf:
+
 libbpf
 ======
 
@@ -7,6 +9,7 @@ libbpf
    :maxdepth: 1
 
    API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
+   program_types
    libbpf_naming_convention
    libbpf_build
 
Documentation/bpf/libbpf/program_types.rst new file +32 -0
new file mode 100644
index 000000000000..04fbb48b8a6a
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+.. _program_types_and_elf:
+
+Program Types and ELF Sections
+==============================
+
+The table below lists the program types, their attach types where relevant and the ELF section
+names supported by libbpf for them. The ELF section names follow these rules:
+
+- ``type`` is an exact match, e.g. ``SEC("socket")``
+- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
+  with a ‘``/``’ separator between ``type`` and ``extras``.
+
+When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
+The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
+for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
+
+..
+  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
+    Program Type,Attach Type,ELF Section Name,Sleepable
+    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv
+   :widths: 40 30 20 10
+   :header-rows: 1
Documentation/bpf/programs.rst +3 -0
index 620eb667ac7a..c99000ab6d9b 100644
@@ -7,3 +7,6 @@ Program Types
    :glob:
 
    prog_*
+
+For a list of all program types, see :ref:`program_types_and_elf` in
+the :ref:`libbpf` documentation.
Signature · 1 line
2.35.1
#9 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

Quoted 4 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

Oh, would be nice to turn all of these into proper but simple Sphinx extensions that take the deps into account in the Sphinx build process. And, of course, would be pure python instead of a combo of Make, shell, and awk.

That's one of the reasons we chose Sphinx originally, to be able to write Sphinx extensions and avoid complicated pipelines.

BR,
Jani.

Quoted 150 lines
---
 Documentation/Makefile                     |  3 +-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 5 files changed, 89 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
Documentation/Makefile +2 -1
index 8a63ef2dcd1c..f007314770e1 100644
@@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
 BUILD_SUBDIRS = \
-	Documentation/userspace-api/media
+	Documentation/userspace-api/media \
+	Documentation/bpf/libbpf
 
 quiet_cmd_build_subdir = SUBDIR  $2
       cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
Documentation/bpf/libbpf/Makefile new file +49 -0
new file mode 100644
index 000000000000..b3dc096c4a96
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Rules to convert BPF program types in tools/lib/bpf/libbpf.c
+# into a .csv file
+
+FILES = program_types.csv
+
+TARGETS := $(addprefix $(BUILDDIR)/, $(FILES))
+
+# Extract program types and properties from the section definitions in libbpf.c such as
+# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv
+#
+# Here is a sample of the generated output that includes .rst formatting:
+#
+#  Program Type,Attach Type,ELF Section Name,Sleepable
+#  ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+$(BUILDDIR)/program_types.csv:	$(srctree)/tools/lib/bpf/libbpf.c
+	$(Q)awk -F'[",[:space:]]+' \
+	'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \
+	/SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \
+	type = "``BPF_PROG_TYPE_" $$4 "``"; \
+	attach = index($$5, "0") ? "" : "``" $$5 "``"; \
+	section = "``" $$3 "``"; \
+	sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \
+	print type "," attach "," section "," sleepable }' \
+	$< > $@
+
+.PHONY: all html epub xml latex linkcheck clean
+
+all: $(BUILDDIR) ${TARGETS}
+	@:
+
+html: all
+epub: all
+xml: all
+latex: all
+linkcheck:
+
+clean:
+	-$(Q)rm -f ${TARGETS} 2>/dev/null
+
+$(BUILDDIR):
+	$(Q)mkdir -p $@
Documentation/bpf/libbpf/index.rst +3 -0
index 3722537d1384..f9b3b252e28f 100644
@@ -1,5 +1,7 @@
 .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+.. _libbpf:
+
 libbpf
 ======
 
@@ -7,6 +9,7 @@ libbpf
    :maxdepth: 1
 
    API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
+   program_types
    libbpf_naming_convention
    libbpf_build
 
Documentation/bpf/libbpf/program_types.rst new file +32 -0
new file mode 100644
index 000000000000..04fbb48b8a6a
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+.. _program_types_and_elf:
+
+Program Types and ELF Sections
+==============================
+
+The table below lists the program types, their attach types where relevant and the ELF section
+names supported by libbpf for them. The ELF section names follow these rules:
+
+- ``type`` is an exact match, e.g. ``SEC("socket")``
+- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
+  with a ‘``/``’ separator between ``type`` and ``extras``.
+
+When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
+The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
+for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
+
+..
+  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
+    Program Type,Attach Type,ELF Section Name,Sleepable
+    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv
+   :widths: 40 30 20 10
+   :header-rows: 1
Documentation/bpf/programs.rst +3 -0
index 620eb667ac7a..c99000ab6d9b 100644
@@ -7,3 +7,6 @@ Program Types
    :glob:
 
    prog_*
+
+For a list of all program types, see :ref:`program_types_and_elf` in
+the :ref:`libbpf` documentation.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
#10 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

Quoted 6 lines · Jani Nikula
Quoted 4 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

Quoted 7 lines

Oh, would be nice to turn all of these into proper but simple Sphinx extensions that take the deps into account in the Sphinx build process. And, of course, would be pure python instead of a combo of Make, shell, and awk.

That's one of the reasons we chose Sphinx originally, to be able to write Sphinx extensions and avoid complicated pipelines.

I could look at this as a followup patch since I would need to learn how to write Sphinx extensions first. It seems that it will require a new reST directive, is that right?

#11 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

Quoted 13 lines · Donald Hunter
Quoted 11 lines · Jani Nikula
Quoted 4 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

Oh, would be nice to turn all of these into proper but simple Sphinx extensions that take the deps into account in the Sphinx build process. And, of course, would be pure python instead of a combo of Make, shell, and awk.

Quoted 6 lines
Quoted 2 lines

That's one of the reasons we chose Sphinx originally, to be able to write Sphinx extensions and avoid complicated pipelines.

I could look at this as a followup patch since I would need to learn how to write Sphinx extensions first. It seems that it will require a new reST directive, is that right?

Correct. It's not very complicated. The extension that handles kernel-doc is under 200 lines, Documentation/sphinx/kerneldoc.py. And a lot of that is just translating directive options to kernel-doc the perl script options.

BR,
Jani.

Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
#12 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

In reply to Donald Hunter
Quoted 7 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

---

[...]

Quoted 16 lines
Documentation/bpf/libbpf/program_types.rst new file +10 -0
new file mode 100644
index 000000000000..04fbb48b8a6a
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+.. _program_types_and_elf:
+
+Program Types and ELF Sections
+==============================
+
+The table below lists the program types, their attach types where relevant and the ELF section
+names supported by libbpf for them. The ELF section names follow these rules:
+

Documentation/bpf/libbpf/program_types.rst line 20

Quoted 10 lines
Documentation/bpf/libbpf/program_types.rst new file +10 -0
+- ``type`` is an exact match, e.g. ``SEC("socket")``
+- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
+  with a ‘``/``’ separator between ``type`` and ``extras``.
+
+When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
+The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
+for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
+
+..
+  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:
                                                                       ^

Typo: s/fomatted/formatted

Documentation/bpf/libbpf/program_types.rst line 32

Quoted 12 lines
Documentation/bpf/libbpf/program_types.rst new file +12 -0
+    Program Type,Attach Type,ELF Section Name,Sleepable
+    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv
+   :widths: 40 30 20 10
+   :header-rows: 1

Nice, I didn't know this formatting trick :-)

Appreciate you are working on this :-)
--Jesper

#13 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

In reply to Donald Hunter
Quoted 131 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. This table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

---
 Documentation/Makefile                     |  3 +-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 5 files changed, 89 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
Documentation/Makefile +2 -1
index 8a63ef2dcd1c..f007314770e1 100644
@@ -66,7 +66,8 @@ I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;
 
 BUILD_SUBDIRS = \
-	Documentation/userspace-api/media
+	Documentation/userspace-api/media \
+	Documentation/bpf/libbpf
 
 quiet_cmd_build_subdir = SUBDIR  $2
       cmd_build_subdir = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=$2 $3
Documentation/bpf/libbpf/Makefile new file +49 -0
new file mode 100644
index 000000000000..b3dc096c4a96
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Rules to convert BPF program types in tools/lib/bpf/libbpf.c
+# into a .csv file
+
+FILES = program_types.csv
+
+TARGETS := $(addprefix $(BUILDDIR)/, $(FILES))
+
+# Extract program types and properties from the section definitions in libbpf.c such as
+# SEC_DEF("socket", SOCKET_FILTER, 0, SEC_NONE) to generate program_types.csv
+#
+# Here is a sample of the generated output that includes .rst formatting:
+#
+#  Program Type,Attach Type,ELF Section Name,Sleepable
+#  ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+#  ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+#  ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+$(BUILDDIR)/program_types.csv:	$(srctree)/tools/lib/bpf/libbpf.c
+	$(Q)awk -F'[",[:space:]]+' \
+	'BEGIN { print "Program Type,Attach Type,ELF Section Name,Sleepable" } \
+	/SEC_DEF\(\"/ && !/SEC_DEPRECATED/ { \
+	type = "``BPF_PROG_TYPE_" $$4 "``"; \
+	attach = index($$5, "0") ? "" : "``" $$5 "``"; \
+	section = "``" $$3 "``"; \
+	sleepable = index($$0, "SEC_SLEEPABLE") ? "Yes" : ""; \
+	print type "," attach "," section "," sleepable }' \
+	$< > $@
+
+.PHONY: all html epub xml latex linkcheck clean
+
+all: $(BUILDDIR) ${TARGETS}
+	@:
+
+html: all
+epub: all
+xml: all
+latex: all
+linkcheck:
+
+clean:
+	-$(Q)rm -f ${TARGETS} 2>/dev/null
+
+$(BUILDDIR):
+	$(Q)mkdir -p $@
Documentation/bpf/libbpf/index.rst +3 -0
index 3722537d1384..f9b3b252e28f 100644
@@ -1,5 +1,7 @@
 .. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 
+.. _libbpf:
+
 libbpf
 ======
 
@@ -7,6 +9,7 @@ libbpf
    :maxdepth: 1
 
    API Documentation <https://libbpf.readthedocs.io/en/latest/api.html>
+   program_types
    libbpf_naming_convention
    libbpf_build
 
Documentation/bpf/libbpf/program_types.rst new file +20 -0
new file mode 100644
index 000000000000..04fbb48b8a6a
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+
+.. _program_types_and_elf:
+
+Program Types and ELF Sections
+==============================
+
+The table below lists the program types, their attach types where relevant and the ELF section
+names supported by libbpf for them. The ELF section names follow these rules:
+
+- ``type`` is an exact match, e.g. ``SEC("socket")``
+- ``type+`` means it can be either exact ``SEC("type")`` or well-formed ``SEC("type/extras")``
+  with a ‘``/``’ separator between ``type`` and ``extras``.
+
+When ``extras`` are specified, they provide details of how to auto-attach the BPF program.
+The format of ``extras`` depends on the program type, e.g. ``SEC("tracepoint/<category>/<name>")``
+for tracepoints or ``SEC("usdt/<path-to-binary>:<usdt_provider>:<usdt_name>")`` for USDT probes.
+
+..
+  program_types.csv is generated from tools/lib/bpf/libbpf.c and is fomatted like this:

Documentation/bpf/libbpf/program_types.rst line 30

Quoted 10 lines
Documentation/bpf/libbpf/program_types.rst new file +10 -0
+    Program Type,Attach Type,ELF Section Name,Sleepable
+    ``BPF_PROG_TYPE_SOCKET_FILTER``,,``socket``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT_OR_MIGRATE``,``sk_reuseport/migrate``,
+    ``BPF_PROG_TYPE_SK_REUSEPORT``,``BPF_SK_REUSEPORT_SELECT``,``sk_reuseport``,
+    ``BPF_PROG_TYPE_KPROBE``,,``kprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe+``,
+    ``BPF_PROG_TYPE_KPROBE``,,``uprobe.s+``,Yes
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv

Oh. You should probably test this with out-of-tree builds.

BR,
Jani.

Quoted 13 lines
Documentation/bpf/libbpf/program_types.rst new file +2 -0
+   :widths: 40 30 20 10
+   :header-rows: 1
Documentation/bpf/programs.rst +3 -0
index 620eb667ac7a..c99000ab6d9b 100644
@@ -7,3 +7,6 @@ Program Types
    :glob:
 
    prog_*
+
+For a list of all program types, see :ref:`program_types_and_elf` in
+the :ref:`libbpf` documentation.
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
#14 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

Quoted 5 lines · Jani Nikula

Documentation/bpf/libbpf/program_types.rst line 30

Quoted 3 lines
Documentation/bpf/libbpf/program_types.rst new file +3 -0
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv

Oh. You should probably test this with out-of-tree builds.

By out of tree, do you mean make htmldocs O=... or something else?

I have tested with O=... whih works for the kernel docs, but I think I will need to rework the location to support the libbpf doc build which has different directory layout.

Thanks,
Donald.

#15 lore raw

Re: [PATCH bpf-next v3 2/2] Add table of BPF program types to libbpf docs

Quoted 9 lines · Donald Hunter
Quoted 5 lines · Jani Nikula

Documentation/bpf/libbpf/program_types.rst line 30

Quoted 3 lines
Documentation/bpf/libbpf/program_types.rst new file +3 -0
+
+.. csv-table:: Program Types and Their ELF Section Names
+   :file: ../../output/program_types.csv

Oh. You should probably test this with out-of-tree builds.

By out of tree, do you mean make htmldocs O=... or something else?

Yeah, O= is what I mean.

BR,
Jani.

Quoted 7 lines

I have tested with O=... whih works for the kernel docs, but I think I will need to rework the location to support the libbpf doc build which has different directory layout.

Thanks,
Donald.

Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Acked-by on the v3 series
#16 lore raw
In reply to Donald Hunter
Quoted 39 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. The table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

Patch 1 adds subdir support to Documentation/Makefile and changes userspace-api/media to use this instead of being a special case.

Patch 2 adds the program_types documentation with a new makefile in the libbpf doc directory to generate program_types.csv

I plan to look at adding info about the format of section "extras" for each program type as a follow-on.

v2 -> v3:
Put program_types after API docs in TOC as suggested by Andrii Nakryiko Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko Include sample of program_types.csv as suggested by Andrii Nakryiko

v1 -> v2:
Automate the generation of program_types.csv as suggested by Andrii Nakryiko.

Donald Hunter (2):
Add subdir support to Documentation makefile
Add table of BPF program types to libbpf docs

 Documentation/Makefile                     | 16 ++++++-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 Documentation/userspace-api/media/Makefile |  2 +
 6 files changed, 103 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
Quoted 3 lines

--
2.35.1

This is marked as Changes Requested, so I presume there are some fixes/updates pending on top of v3? BPF CI should probably be done as a follow up, though.

But otherwise looks good to me and I appreciate the effort to improve libbpf's documentations.

#17 lore raw
Quoted 40 lines · Andrii Nakryiko
Quoted 39 lines · Donald Hunter

Extend the libbpf documentation with a table of program types, attach points and ELF section names. The table uses data from program_types.csv which is generated from tools/lib/bpf/libbpf.c during the documentation build.

Patch 1 adds subdir support to Documentation/Makefile and changes userspace-api/media to use this instead of being a special case.

Patch 2 adds the program_types documentation with a new makefile in the libbpf doc directory to generate program_types.csv

I plan to look at adding info about the format of section "extras" for each program type as a follow-on.

v2 -> v3:
Put program_types after API docs in TOC as suggested by Andrii Nakryiko Fix formatting as reported by Andrii Nakryiko
Include USDT extras example as suggested by Andrii Nakryiko Include sample of program_types.csv as suggested by Andrii Nakryiko

v1 -> v2:
Automate the generation of program_types.csv as suggested by Andrii Nakryiko.

Donald Hunter (2):
Add subdir support to Documentation makefile
Add table of BPF program types to libbpf docs

 Documentation/Makefile                     | 16 ++++++-
 Documentation/bpf/libbpf/Makefile          | 49 ++++++++++++++++++++++
 Documentation/bpf/libbpf/index.rst         |  3 ++
 Documentation/bpf/libbpf/program_types.rst | 32 ++++++++++++++
 Documentation/bpf/programs.rst             |  3 ++
 Documentation/userspace-api/media/Makefile |  2 +
 6 files changed, 103 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/bpf/libbpf/Makefile
 create mode 100644 Documentation/bpf/libbpf/program_types.rst
Quoted 7 lines
Quoted 3 lines

--
2.35.1

This is marked as Changes Requested, so I presume there are some fixes/updates pending on top of v3? BPF CI should probably be done as a follow up, though.

Yes, there is a typo reported by Jesper Brouer that will require a v4.

Quoted 4 lines

But otherwise looks good to me and I appreciate the effort to improve libbpf's documentations.

This patchset will break the documentation build in the libbpf GitHub repository when Documentation/bpf/libbpf gets synced to there.

At a minimum, I think I would need to refactor this patchset so that it is easier to integrate the .csv generation into the libbpf GitHub documentation build. But a simpler way forward might be to move this entirely to the libbpf Github repository and then just add a readthedocs URL to Documentation/bpf/libbpf/index.rst, the same as currently done for the API docs.