Skip to contentSkip to conversation

Autogenerating libbpf API documentation

PATCH Superseded grantseltzer 2 participants 5 messages Jun 17, 2021 – Jun 18, 2021

Subsystems (3)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Autogenerating libbpf API documentation

This patch series is meant to start the initiative to document libbpf. It includes .rst files which are text documentation describing building, API naming convention, as well as an index to generated API documentation.

In this approach the generated API documentation is enabled by the kernels existing kernel documentation system which uses sphinx. The resulting docs would then be synced to kernel.org/doc

You can test this by running make htmldocs and serving the html in Documentation/output. Since libbpf does not yet have comments in kernel doc format, see kernel.org/doc/html/latest/doc-guide/kernel-doc.html for an example so you can test this.

The advantage of this approach is to use the existing sphinx infrastructure that the kernel has, and have libbpf docs in the same place as everything else.

The current plan is to have the libbpf mirror sync the generated docs and version them based on the libbpf releases which are cut on github.

grantseltzer (1):
Add documentation for libbpf including API autogen

 Documentation/bpf/index.rst                   | 13 +++++++
 Documentation/bpf/libbpf.rst                  | 14 +++++++
 Documentation/bpf/libbpf_api.rst              | 27 ++++++++++++++
 Documentation/bpf/libbpf_build.rst            | 37 +++++++++++++++++++
 .../bpf/libbpf_naming_convention.rst          | 32 +++++++---------
 5 files changed, 104 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/bpf/libbpf.rst
 create mode 100644 Documentation/bpf/libbpf_api.rst
 create mode 100644 Documentation/bpf/libbpf_build.rst
 rename tools/lib/bpf/README.rst => Documentation/bpf/libbpf_naming_convention.rst (89%)
Signature · 1 line
2.31.1
#1 lore raw

PATCH v3 1/1 Add documentation for libbpf including API autogen

This adds rst files containing documentation for libbpf. This includes the addition of libbpf_api.rst which pulls comment documentation from header files in libbpf under tools/lib/bpf/. The comment docs would be of the standard kernel doc format.

---
 Documentation/bpf/index.rst                   | 13 +++++++
 Documentation/bpf/libbpf.rst                  | 14 +++++++
 Documentation/bpf/libbpf_api.rst              | 27 ++++++++++++++
 Documentation/bpf/libbpf_build.rst            | 37 +++++++++++++++++++
 .../bpf/libbpf_naming_convention.rst          | 32 +++++++---------
 5 files changed, 104 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/bpf/libbpf.rst
 create mode 100644 Documentation/bpf/libbpf_api.rst
 create mode 100644 Documentation/bpf/libbpf_build.rst
 rename tools/lib/bpf/README.rst => Documentation/bpf/libbpf_naming_convention.rst (89%)
Documentation/bpf/index.rst +13 -0
index a702f67dd..44f646735 100644
@@ -12,6 +12,19 @@ BPF instruction-set.
 The Cilium project also maintains a `BPF and XDP Reference Guide`_
 that goes into great technical depth about the BPF Architecture.
 
+libbpf
+======
+
+Libbpf is a userspace library for loading and interacting with bpf programs.
+
+.. toctree::
+   :maxdepth: 1
+
+   libbpf
+   libbpf_api
+   libbpf_build
+   libbpf_naming_convention
+
 BPF Type Format (BTF)
 =====================
 
Documentation/bpf/libbpf.rst new file +14 -0
new file mode 100644
index 000000000..2e62cadee
@@ -0,0 +1,14 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+libbpf
+======
+
+This is documentation for libbpf, a userspace library for loading and
+interacting with bpf programs.
+
+All general BPF questions, including kernel functionality, libbpf APIs and
+their application, should be sent to [email protected] mailing list.
+You can `subscribe <http://vger.kernel.org/vger-lists.html#bpf>`_ to the
+mailing list search its `archive <https://lore.kernel.org/bpf/>`_.
+Please search the archive before asking new questions. It very well might
+be that this was already addressed or answered before.
Documentation/bpf/libbpf_api.rst new file +27 -0
new file mode 100644
index 000000000..b54bd7470
@@ -0,0 +1,27 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+API
+===
+
+This documentation is autogenerated from header files in libbpf, tools/lib/bpf
+
+.. kernel-doc:: tools/lib/bpf/libbpf.h
+   :internal:
+
+.. kernel-doc:: tools/lib/bpf/bpf.h
+   :internal:
+
+.. kernel-doc:: tools/lib/bpf/btf.h
+   :internal:
+
+.. kernel-doc:: tools/lib/bpf/xsk.h
+   :internal:
+
+.. kernel-doc:: tools/lib/bpf/bpf_tracing.h
+   :internal:
+
+.. kernel-doc:: tools/lib/bpf/bpf_core_read.h
+   :internal:
+
+.. kernel-doc:: tools/lib/bpf/bpf_endian.h
+   :internal:
\ No newline at end of file
Documentation/bpf/libbpf_build.rst new file +37 -0
new file mode 100644
index 000000000..584be90f1
@@ -0,0 +1,37 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+Building libbpf
+===============
+
+libelf and zlib are internal dependencies of libbpf and thus are required to link
+against and must be installed on the system for applications to work.
+pkg-config is used by default to find libelf, and the program called
+can be overridden with PKG_CONFIG.
+
+If using pkg-config at build time is not desired, it can be disabled by
+setting NO_PKG_CONFIG=1 when calling make.
+
+To build both static libbpf.a and shared libbpf.so:
+
+.. code-block:: bash
+
+    $ cd src
+    $ make
+
+To build only static libbpf.a library in directory build/ and install them
+together with libbpf headers in a staging directory root/:
+
+.. code-block:: bash
+
+    $ cd src
+    $ mkdir build root
+    $ BUILD_STATIC_ONLY=y OBJDIR=build DESTDIR=root make install
+
+To build both static libbpf.a and shared libbpf.so against a custom libelf
+dependency installed in /build/root/ and install them together with libbpf
+headers in a build directory /build/root/:
+
+.. code-block:: bash
+
+    $ cd src
+    $ PKG_CONFIG_PATH=/build/root/lib64/pkgconfig DESTDIR=/build/root make
\ No newline at end of file
Documentation/bpf/libbpf_naming_convention.rst renamed from tools/lib/bpf/README.rst +13 -19
similarity index 89%
rename from tools/lib/bpf/README.rst
rename to Documentation/bpf/libbpf_naming_convention.rst
index 8928f7787..b6dc5c592 100644
@@ -1,7 +1,7 @@
-.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+.. SPDX-License-Identifier: GPL-2.0
 
-libbpf API naming convention
-============================
+API naming convention
+=====================
 
 libbpf API provides access to a few logically separated groups of
 functions and types. Every group has its own naming convention
@@ -10,14 +10,14 @@ new function or type is added to keep libbpf API clean and consistent.
 
 All types and functions provided by libbpf API should have one of the
 following prefixes: ``bpf_``, ``btf_``, ``libbpf_``, ``xsk_``,
-``perf_buffer_``.
+``btf_dump_``, ``ring_buffer_``, ``perf_buffer_``.
 
 System call wrappers
 --------------------
 
 System call wrappers are simple wrappers for commands supported by
 sys_bpf system call. These wrappers should go to ``bpf.h`` header file
-and map one-on-one to corresponding commands.
+and map one to one to corresponding commands.
 
 For example ``bpf_map_lookup_elem`` wraps ``BPF_MAP_LOOKUP_ELEM``
 command of sys_bpf, ``bpf_prog_attach`` wraps ``BPF_PROG_ATTACH``, etc.
@@ -49,10 +49,6 @@ object, ``bpf_object``, double underscore and ``open`` that defines the
 purpose of the function to open ELF file and create ``bpf_object`` from
 it.
 
-Another example: ``bpf_program__load`` is named for corresponding
-object, ``bpf_program``, that is separated from other part of the name
-by double underscore.
-
 All objects and corresponding functions other than BTF related should go
 to ``libbpf.h``. BTF types and functions should go to ``btf.h``.
 
@@ -72,11 +68,7 @@ of both low-level ring access functions and high-level configuration
 functions. These can be mixed and matched. Note that these functions
 are not reentrant for performance reasons.
 
-Please take a look at Documentation/networking/af_xdp.rst in the Linux
-kernel source tree on how to use XDP sockets and for some common
-mistakes in case you do not get any traffic up to user space.
-
-libbpf ABI
+ABI
 ==========
 
 libbpf can be both linked statically or used as DSO. To avoid possible
@@ -116,7 +108,8 @@ This bump in ABI version is at most once per kernel development cycle.
 
 For example, if current state of ``libbpf.map`` is:
 
-.. code-block::
+.. code-block:: c
+
         LIBBPF_0.0.1 {
         	global:
                         bpf_func_a;
@@ -128,7 +121,8 @@ For example, if current state of ``libbpf.map`` is:
 , and a new symbol ``bpf_func_c`` is being introduced, then
 ``libbpf.map`` should be changed like this:
 
-.. code-block::
+.. code-block:: c
+
         LIBBPF_0.0.1 {
         	global:
                         bpf_func_a;
@@ -148,7 +142,7 @@ Format of version script and ways to handle ABI changes, including
 incompatible ones, described in details in [1].
 
 Stand-alone build
-=================
+-------------------
 
 Under https://github.com/libbpf/libbpf there is a (semi-)automated
 mirror of the mainline's version of libbpf for a stand-alone build.
@@ -157,12 +151,12 @@ However, all changes to libbpf's code base must be upstreamed through
 the mainline kernel tree.
 
 License
-=======
+-------------------
 
 libbpf is dual-licensed under LGPL 2.1 and BSD 2-Clause.
 
 Links
-=====
+-------------------
 
 [1] https://www.akkadia.org/drepper/dsohowto.pdf
     (Chapter 3. Maintaining APIs and ABIs).
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH bpf-next v3 1/1] Add documentation for libbpf including API autogen

Quoted 6 lines · grantseltzer

This adds rst files containing documentation for libbpf. This includes the addition of libbpf_api.rst which pulls comment documentation from header files in libbpf under tools/lib/bpf/. The comment docs would be of the standard kernel doc format.

Quoted 6 lines
---
 Documentation/bpf/index.rst                   | 13 +++++++
 Documentation/bpf/libbpf.rst                  | 14 +++++++
 Documentation/bpf/libbpf_api.rst              | 27 ++++++++++++++
 Documentation/bpf/libbpf_build.rst            | 37 +++++++++++++++++++

Didn't we agree to have docs under Documentation/bpf/libbpf? That should make it clear that each is libbpf-specific and probably would make copying/syncing easier. Plus it will be a libbpf sub-section in the docs, no?

Quoted 37 lines
 .../bpf/libbpf_naming_convention.rst          | 32 +++++++---------
 5 files changed, 104 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/bpf/libbpf.rst
 create mode 100644 Documentation/bpf/libbpf_api.rst
 create mode 100644 Documentation/bpf/libbpf_build.rst
 rename tools/lib/bpf/README.rst => Documentation/bpf/libbpf_naming_convention.rst (89%)
Documentation/bpf/index.rst +13 -0
index a702f67dd..44f646735 100644
@@ -12,6 +12,19 @@ BPF instruction-set.
 The Cilium project also maintains a `BPF and XDP Reference Guide`_
 that goes into great technical depth about the BPF Architecture.
 
+libbpf
+======
+
+Libbpf is a userspace library for loading and interacting with bpf programs.
+
+.. toctree::
+   :maxdepth: 1
+
+   libbpf
+   libbpf_api
+   libbpf_build
+   libbpf_naming_convention
+
 BPF Type Format (BTF)
 =====================
 
Documentation/bpf/libbpf.rst new file +0 -0
new file mode 100644
index 000000000..2e62cadee

Documentation/bpf/libbpf.rst line 1

Quoted 1 line
Documentation/bpf/libbpf.rst new file +1 -0
@@ -0,0 +1,14 @@
+.. SPDX-License-Identifier: GPL-2.0

Should we use dual-license LGPL-2.1 OR BSD-2-Clause like the rest of libbpf?

Quoted 7 lines
Documentation/bpf/libbpf.rst new file +7 -0
+
+libbpf
+======
+
+This is documentation for libbpf, a userspace library for loading and
+interacting with bpf programs.
+

[...]

Documentation/bpf/libbpf_naming_convention.rst line 1

Quoted 13 lines
Documentation/bpf/libbpf.rst new file +2 -0
+    $ cd src
+    $ PKG_CONFIG_PATH=/build/root/lib64/pkgconfig DESTDIR=/build/root make
\ No newline at end of file
Documentation/bpf/libbpf_naming_convention.rst renamed from tools/lib/bpf/README.rst +1 -1
similarity index 89%
rename from tools/lib/bpf/README.rst
rename to Documentation/bpf/libbpf_naming_convention.rst
index 8928f7787..b6dc5c592 100644
@@ -1,7 +1,7 @@
-.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+.. SPDX-License-Identifier: GPL-2.0

I don't think we can just easily re-license without asking original contributor. But see above, I think we should stick to the dual-license to stay consistent with libbpf sources?

[...]

#3 lore raw

Re: [PATCH bpf-next v3 1/1] Add documentation for libbpf including API autogen

Quoted 15 lines · Andrii Nakryiko
Quoted 6 lines · grantseltzer

This adds rst files containing documentation for libbpf. This includes the addition of libbpf_api.rst which pulls comment documentation from header files in libbpf under tools/lib/bpf/. The comment docs would be of the standard kernel doc format.

Quoted 6 lines
---
 Documentation/bpf/index.rst                   | 13 +++++++
 Documentation/bpf/libbpf.rst                  | 14 +++++++
 Documentation/bpf/libbpf_api.rst              | 27 ++++++++++++++
 Documentation/bpf/libbpf_build.rst            | 37 +++++++++++++++++++
Quoted 4 lines

Didn't we agree to have docs under Documentation/bpf/libbpf? That should make it clear that each is libbpf-specific and probably would make copying/syncing easier. Plus it will be a libbpf sub-section in the docs, no?

Ah sure, that works.

Quoted 61 lines
Quoted 38 lines
 .../bpf/libbpf_naming_convention.rst          | 32 +++++++---------
 5 files changed, 104 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/bpf/libbpf.rst
 create mode 100644 Documentation/bpf/libbpf_api.rst
 create mode 100644 Documentation/bpf/libbpf_build.rst
 rename tools/lib/bpf/README.rst => Documentation/bpf/libbpf_naming_convention.rst (89%)
Documentation/bpf/index.rst +13 -0
index a702f67dd..44f646735 100644
@@ -12,6 +12,19 @@ BPF instruction-set.
 The Cilium project also maintains a `BPF and XDP Reference Guide`_
 that goes into great technical depth about the BPF Architecture.
 
+libbpf
+======
+
+Libbpf is a userspace library for loading and interacting with bpf programs.
+
+.. toctree::
+   :maxdepth: 1
+
+   libbpf
+   libbpf_api
+   libbpf_build
+   libbpf_naming_convention
+
 BPF Type Format (BTF)
 =====================
 
Documentation/bpf/libbpf.rst new file +0 -0
new file mode 100644
index 000000000..2e62cadee

Documentation/bpf/libbpf.rst line 1

Quoted 1 line
Documentation/bpf/libbpf.rst new file +1 -0
@@ -0,0 +1,14 @@
+.. SPDX-License-Identifier: GPL-2.0

Should we use dual-license LGPL-2.1 OR BSD-2-Clause like the rest of libbpf?

Quoted 7 lines
Documentation/bpf/libbpf.rst new file +7 -0
+
+libbpf
+======
+
+This is documentation for libbpf, a userspace library for loading and
+interacting with bpf programs.
+

[...]

Quoted 9 lines
Documentation/bpf/libbpf.rst new file +2 -0
+    $ cd src
+    $ PKG_CONFIG_PATH=/build/root/lib64/pkgconfig DESTDIR=/build/root make
\ No newline at end of file
Documentation/bpf/libbpf_naming_convention.rst renamed from tools/lib/bpf/README.rst +0 -1
similarity index 89%
rename from tools/lib/bpf/README.rst
rename to Documentation/bpf/libbpf_naming_convention.rst
index 8928f7787..b6dc5c592 100644
--- a/tools/lib/bpf/README.rst
Quoted 8 lines

Documentation/bpf/libbpf_naming_convention.rst line 1

Quoted 4 lines

+++ b/Documentation/bpf/libbpf_naming_convention.rst

Documentation/bpf/libbpf_naming_convention.rst renamed from tools/lib/bpf/README.rst +1 -1
@@ -1,7 +1,7 @@
-.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+.. SPDX-License-Identifier: GPL-2.0

I don't think we can just easily re-license without asking original contributor. But see above, I think we should stick to the dual-license to stay consistent with libbpf sources?

This change was not at all intentional. I'll change it back to the dual license.

Quoted 3 lines

[...]

#4 lore raw
In reply to grantseltzer
Quoted 18 lines · grantseltzer

This patch series is meant to start the initiative to document libbpf. It includes .rst files which are text documentation describing building, API naming convention, as well as an index to generated API documentation.

In this approach the generated API documentation is enabled by the kernels existing kernel documentation system which uses sphinx. The resulting docs would then be synced to kernel.org/doc

You can test this by running make htmldocs and serving the html in Documentation/output. Since libbpf does not yet have comments in kernel doc format, see kernel.org/doc/html/latest/doc-guide/kernel-doc.html for an example so you can test this.

The advantage of this approach is to use the existing sphinx infrastructure that the kernel has, and have libbpf docs in the same place as everything else.

Quoted 6 lines

The current plan is to have the libbpf mirror sync the generated docs and version them based on the libbpf releases which are cut on github.

grantseltzer (1):
Add documentation for libbpf including API autogen

You don't need the cover letter if you are submitting a single patch. Just put all of the context in cover letter into the patch description.

Quoted 13 lines
 Documentation/bpf/index.rst                   | 13 +++++++
 Documentation/bpf/libbpf.rst                  | 14 +++++++
 Documentation/bpf/libbpf_api.rst              | 27 ++++++++++++++
 Documentation/bpf/libbpf_build.rst            | 37 +++++++++++++++++++
 .../bpf/libbpf_naming_convention.rst          | 32 +++++++---------
 5 files changed, 104 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/bpf/libbpf.rst
 create mode 100644 Documentation/bpf/libbpf_api.rst
 create mode 100644 Documentation/bpf/libbpf_build.rst
 rename tools/lib/bpf/README.rst => Documentation/bpf/libbpf_naming_convention.rst (89%)

--
2.31.1