Skip to contentSkip to conversation

doc: DEVMAPs and XDP_REDIRECT

PATCH Superseded [email protected] 4 participants 6 messages Oct 17, 2022 – Oct 25, 2022

Subsystems (2)

v3 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v3 cover doc: DEVMAPs and XDP_REDIRECT

From: Maryam Tahhan <[email protected]>

Add documentation for BPF_MAP_TYPE_DEVMAP and
BPF_MAP_TYPE_DEVMAP_HASH including kernel version
introduced, usage and examples.

Add documentation that describes XDP_REDIRECT.

v2->v3:
- Fixed indentations in usage section to exclude non note text.
- Replace links to selftest with actual paths.

v1->v2:
- Separate xdp_redirect documentation to its own file.
- Clean up and simplify examples and usage function descriptions.

Maryam Tahhan (1):
doc: DEVMAPs and XDP_REDIRECT

 Documentation/bpf/index.rst      |   1 +
 Documentation/bpf/map_devmap.rst | 189 +++++++++++++++++++++++++++++++
 Documentation/bpf/redirect.rst   |  46 ++++++++
 3 files changed, 236 insertions(+)
 create mode 100644 Documentation/bpf/map_devmap.rst
 create mode 100644 Documentation/bpf/redirect.rst
Signature · 1 line
2.35.3
#1 lore raw

PATCH v3 1/1 doc: DEVMAPs and XDP_REDIRECT

From: Maryam Tahhan <[email protected]>

Add documentation for BPF_MAP_TYPE_DEVMAP and
BPF_MAP_TYPE_DEVMAP_HASH including kernel version
introduced, usage and examples.

Add documentation that describes XDP_REDIRECT.

---
 Documentation/bpf/index.rst      |   1 +
 Documentation/bpf/map_devmap.rst | 189 +++++++++++++++++++++++++++++++
 Documentation/bpf/redirect.rst   |  46 ++++++++
 3 files changed, 236 insertions(+)
 create mode 100644 Documentation/bpf/map_devmap.rst
 create mode 100644 Documentation/bpf/redirect.rst
Documentation/bpf/index.rst +1 -0
index 1b50de1983ee..1088d44634d6 100644
@@ -29,6 +29,7 @@ that goes into great technical depth about the BPF Architecture.
    clang-notes
    linux-notes
    other
+   redirect
 
 .. only::  subproject and html
 
Documentation/bpf/map_devmap.rst new file +189 -0

189 lines not on this page · raw message

Documentation/bpf/redirect.rst new file +46 -0

46 lines not on this page · raw message

Signature · 1 line
2.35.3
#2 lore raw
Quoted 9 lines · [email protected] writes:
Documentation/bpf/redirect.rst new file +3 -0
new file mode 100644
index 000000000000..5a0377a67ff0
@@ -0,0 +1,46 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+.. Copyright (C) 2022 Red Hat, Inc.
+

Documentation/bpf/redirect.rst line 13

Quoted 10 lines
Documentation/bpf/redirect.rst new file +10 -0
+============
+XDP_REDIRECT
+============
+
+XDP_REDIRECT works by a three-step process, implemented as follows:
+
+1. The ``bpf_redirect()`` and ``bpf_redirect_map()`` helpers will lookup the
+   target of the redirect and store it (along with some other metadata) in a
+   per-CPU ``struct bpf_redirect_info``. This is where the maps above come into
+   play.

Can you remove the last sentence above. Instead, maybe mention that the lookup happens in the provided map which has to be one of the supported map types.

Quoted 32 lines
Documentation/bpf/redirect.rst new file +32 -0
+2. When the program returns the ``XDP_REDIRECT`` return code, the driver will
+   call ``xdp_do_redirect()`` which will use the information in ``struct
+   bpf_redirect_info`` to actually enqueue the frame into a map type-specific
+   bulk queue structure.
+
+3. Before exiting its NAPI poll loop, the driver will call ``xdp_do_flush()``,
+   which will flush all the different bulk queues, thus completing the
+   redirect.
+
+Pointers to the map entries will be kept around for this whole sequence of
+steps, protected by RCU. However, there is no top-level ``rcu_read_lock()`` in
+the core code; instead, the RCU protection relies on everything happening
+inside a single NAPI poll sequence.
+
+.. note::
+    Not all drivers support transmitting frames after a redirect, and for
+    those that do, not all of them support non-linear frames. Non-linear xdp
+    bufs/frames are bufs/frames that contain more than one fragment.
+
+XDP_REDIRECT works with the following map types:
+
+- BPF_MAP_TYPE_DEVMAP
+- BPF_MAP_TYPE_DEVMAP_HASH
+- BPF_MAP_TYPE_CPUMAP
+- BPF_MAP_TYPE_XSKMAP
+
+For more information on these maps, please see the specific map documentation.
+
+References
+===========
+
+- https://elixir.bootlin.com/linux/latest/source/net/core/filter.c#L4106
#3 lore raw
In reply to [email protected]

First of all, I'm super happy that we are getting documentation added for this.

Comments inlined below.

Documentation/bpf/redirect.rst

Quoted 5 lines · On 17/10/2022 11.47, [email protected] wrote:
Documentation/bpf/redirect.rst new file +0 -0
new file mode 100644
index 000000000000..5a0377a67ff0

Naming the file 'redirect.rst' is that in anticipating that TC-BPF also support invoking the bpf_redirect helper?

IMHO we should remember to *also* promote TC-BPF redirect, and it would likely be good to have this in same file with XDP-redirect so end-users see this.

Quoted 13 lines
Documentation/bpf/redirect.rst new file +12 -0
@@ -0,0 +1,46 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+.. Copyright (C) 2022 Red Hat, Inc.
+
+============
+XDP_REDIRECT
+============
+
+XDP_REDIRECT works by a three-step process, implemented as follows:
+
+1. The ``bpf_redirect()`` and ``bpf_redirect_map()`` helpers will lookup the
+   target of the redirect and store it (along with some other metadata) in a
+   per-CPU ``struct bpf_redirect_info``. This is where the maps above come into

Documentation/bpf/redirect.rst line 22

Quoted 10 lines
Documentation/bpf/redirect.rst new file +10 -0
+   play.
+
+2. When the program returns the ``XDP_REDIRECT`` return code, the driver will
+   call ``xdp_do_redirect()`` which will use the information in ``struct
+   bpf_redirect_info`` to actually enqueue the frame into a map type-specific
+   bulk queue structure.
+
+3. Before exiting its NAPI poll loop, the driver will call ``xdp_do_flush()``,
+   which will flush all the different bulk queues, thus completing the
+   redirect.

Is this text more or less copied from net/core/filter.c ?

I will suggest directly including this from the code via the DOC text trick. (note I've added these DOC tags in other XDP + page_pool code, but not fully utilized these yet).

Documentation/bpf/redirect.rst line 33

Quoted 10 lines
Documentation/bpf/redirect.rst new file +10 -0
+Pointers to the map entries will be kept around for this whole sequence of
+steps, protected by RCU. However, there is no top-level ``rcu_read_lock()`` in
+the core code; instead, the RCU protection relies on everything happening
+inside a single NAPI poll sequence.
+
+.. note::
+    Not all drivers support transmitting frames after a redirect, and for
+    those that do, not all of them support non-linear frames. Non-linear xdp
+    bufs/frames are bufs/frames that contain more than one fragment.
+

I would like for us to extend this redirect.rst document with information on how to troubleshoot when XDP-redirect "silently" drops packets.

Above note it one issue (but not visible to readers).
Plus we should describe how to catch these silent drops, via tracepoints and even point to xdpdump tool.

I recently helped someone on Slack debug a XDP redirect issue.
During this session I wrote some bpftrace oneliners, that I added to XDP-tutorial sub-README[1]

[1] https://github.com/xdp-project/xdp-tutorial/blob/master/tracing02-xdp-monitor/README.org

Documentation/bpf/redirect.rst

Quoted 13 lines
Documentation/bpf/redirect.rst new file +13 -0
+XDP_REDIRECT works with the following map types:
+
+- BPF_MAP_TYPE_DEVMAP
+- BPF_MAP_TYPE_DEVMAP_HASH
+- BPF_MAP_TYPE_CPUMAP
+- BPF_MAP_TYPE_XSKMAP
+
+For more information on these maps, please see the specific map documentation.
+
+References
+===========
+
+-https://elixir.bootlin.com/linux/latest/source/net/core/filter.c#L4106

I don't think this reference with a line-number will be stable.

--Jesper

#4 lore raw
Quoted 14 lines · Jesper Dangaard Brouer

First of all, I'm super happy that we are getting documentation added for this.

Comments inlined below.

Documentation/bpf/redirect.rst

Quoted 5 lines · On 17/10/2022 11.47, [email protected] wrote:
Documentation/bpf/redirect.rst new file +0 -0
new file mode 100644
index 000000000000..5a0377a67ff0

Naming the file 'redirect.rst' is that in anticipating that TC-BPF also support invoking the bpf_redirect helper?

Quoted 2 lines

IMHO we should remember to *also* promote TC-BPF redirect, and it would likely be good to have this in same file with XDP-redirect so end-users see this.

So I will leave the name as is...

Quoted 33 lines
Quoted 22 lines
Documentation/bpf/redirect.rst new file +10 -0
@@ -0,0 +1,46 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+.. Copyright (C) 2022 Red Hat, Inc.
+
+============
+XDP_REDIRECT
+============
+
+XDP_REDIRECT works by a three-step process, implemented as follows:
+
+1. The ``bpf_redirect()`` and ``bpf_redirect_map()`` helpers will 

lookup the
+   target of the redirect and store it (along with some other metadata) in a
+   per-CPU ``struct bpf_redirect_info``. This is where the maps above come into
+   play.
+
+2. When the program returns the ``XDP_REDIRECT`` return code, the driver will
+   call ``xdp_do_redirect()`` which will use the information in ``struct

Quoted 8 lines

+   bpf_redirect_info`` to actually enqueue the frame into a map type-specific
+   bulk queue structure.
+
+3. Before exiting its NAPI poll loop, the driver will call ``xdp_do_flush()``,
+   which will flush all the different bulk queues, thus completing the +   redirect.

Is this text more or less copied from net/core/filter.c ?

Quoted 4 lines

I will suggest directly including this from the code via the DOC text trick.  (note I've added these DOC tags in other XDP + page_pool code, but not fully utilized these yet).

Ok, I will update this. I had the v5 sent in before I saw your email.

Quoted 21 lines
Quoted 11 lines

+Pointers to the map entries will be kept around for this whole sequence of
+steps, protected by RCU. However, there is no top-level ``rcu_read_lock()`` in
+the core code; instead, the RCU protection relies on everything happening

Documentation/bpf/redirect.rst new file +4 -0
+inside a single NAPI poll sequence.
+
+.. note::
+    Not all drivers support transmitting frames after a redirect, and 
Quoted 5 lines

for
+    those that do, not all of them support non-linear frames. Non-linear xdp
+    bufs/frames are bufs/frames that contain more than one fragment. +

I would like for us to extend this redirect.rst document with information on how to troubleshoot when XDP-redirect "silently" drops packets.

Quoted 9 lines

Above note it one issue (but not visible to readers).
Plus we should describe how to catch these silent drops, via tracepoints and even point to xdpdump tool.

I recently helped someone on Slack debug a XDP redirect issue.
During this session I wrote some bpftrace oneliners, that I added to XDP-tutorial sub-README[1]

[1] https://github.com/xdp-project/xdp-tutorial/blob/master/tracing02-xdp-monitor/README.org

Ok, I will see what I can do.

Quoted 17 lines
Quoted 10 lines
Documentation/bpf/redirect.rst new file +8 -0
+XDP_REDIRECT works with the following map types:
+
+- BPF_MAP_TYPE_DEVMAP
+- BPF_MAP_TYPE_DEVMAP_HASH
+- BPF_MAP_TYPE_CPUMAP
+- BPF_MAP_TYPE_XSKMAP
+
+For more information on these maps, please see the specific map 

documentation.

Documentation/bpf/redirect.rst

Quoted 5 lines
Documentation/bpf/redirect.rst new file +5 -0
+
+References
+===========
+
+-https://elixir.bootlin.com/linux/latest/source/net/core/filter.c#L4106

I don't think this reference with a line-number will be stable.

Yep, will move to the doc reference as you suggested earlier.

Quoted 3 lines

--Jesper

#5 lore raw
Quoted 19 lines · Maryam Tahhan
Quoted 17 lines · Jesper Dangaard Brouer

First of all, I'm super happy that we are getting documentation added for this.

Comments inlined below.

Documentation/bpf/redirect.rst

Quoted 6 lines · On 17/10/2022 11.47, [email protected] wrote:
Documentation/bpf/redirect.rst new file +0 -0
new file mode 100644
index 000000000000..5a0377a67ff0

Naming the file 'redirect.rst' is that in anticipating that TC-BPF also support invoking the bpf_redirect helper?

Quoted 6 lines
Quoted 4 lines

IMHO we should remember to *also* promote TC-BPF redirect, and it would likely be good to have this in same file with XDP-redirect so end-users see this.

So I will leave the name as is...

Please fold it into xdp doc that describes all return codes. It's weird to have a separate file just for one of the return values and not for the others.