BPF link support for tc BPF programs
Subsystems (10)
Daniel Borkmann
maintainerv4 cover BPF link support for tc BPF programs
This series adds BPF link support for tc BPF programs. We initially presented the motivation, related work and design at last year's LPC conference in the networking & BPF track [0], and a recent update on our progress of the rework during this year's LSF/MM/BPF summit [1]. The main changes are in first two patches and the last two have an extensive batch of test cases we developed along with it, please see individual patches for details. We tested this series with tc-testing selftest suite as well as BPF CI/selftests. Thanks!
v3 -> v4:
- Fix bpftool output to display tcx/{ingress,egress} (Stan)
- Documentation around API, BPF_MPROG_* return codes and locking expectations (Stan, Alexei)
- Change _after and _before to have the same semantics for return value (Alexei)
- Rework mprog initialization and move allocation/free one layer up into tcx to simplify the code (Stan)
- Add comment on synchronize_rcu and parent->ref (Stan)
- Add comment on bpf_mprog_pos_() helpers wrt target position (Stan)
v2 -> v3:
- Removal of BPF_F_FIRST/BPF_F_LAST from control UAPI (Toke, Stan)
- Along with that full rework of bpf_mprog internals to simplify dependency management, looks much nicer now imho
- Just single bpf_mprog_cp instead of two (Andrii)
- atomic64_t for revision counter (Andrii)
- Evaluate target position and reject on conflicts (Andrii)
- Keep track of actual count in bpf_mprob_bundle (Andrii)
- Make combo of REPLACE and BEFORE/AFTER work (Andrii)
- Moved miniq as first struct member (Jamal)
- Rework tcx_link_attach with regards to rtnl (Jakub, Andrii)
- Moved wrappers after bpf_prog_detach_ops (Andrii)
- Removed union for relative_fd and friends for opts and link in libbpf (Andrii)
- Add doc comments to attach/detach/query libbpf APIs (Andrii)
- Dropped SEC_ATTACHABLE_OPT (Andrii)
- Add an OPTS_ZEROED check to bpf_link_create (Andrii)
- Keep opts as the last argument in bpf_program_attach_fd (Andrii)
- Rework bpf_program_attach_fd (Andrii)
- Remove OPTS_GET before we checked OPTS_VALID in bpf_program__attach_tcx (Andrii)
- Add size_t :0; to prevent compiler from leaving garbage (Andrii)
- Add helper macro to clear opts structs which I found useful when writing tests
- Rework of both opts and link test cases to accommodate for changes
v1 -> v2:
- Rework of almost entire series to remove prio from UAPI and switch to better control directives BPF_F_FIRST/BPF_F_LAST/BPF_F_BEFORE/ BPF_F_AFTER (Alexei, Toke, Stan, Andrii)
- Addition of big test suite to cover all corner cases
[0] https://lpc.events/event/16/contributions/1353/
[1] http://vger.kernel.org/bpfconf2023_material/tcx_meta_netdev_borkmann.pdf
Daniel Borkmann (8):
bpf: Add generic attach/detach/query API for multi-progs
bpf: Add fd-based tcx multi-prog infra with link support
libbpf: Add opts-based attach/detach/query API for tcx
libbpf: Add link-based API for tcx
libbpf: Add helper macro to clear opts structs
bpftool: Extend net dump with tcx progs selftests/bpf: Add mprog API tests for BPF tcx opts selftests/bpf: Add mprog API tests for BPF tcx links
MAINTAINERS | 5 +- include/linux/bpf_mprog.h | 352 +++ include/linux/netdevice.h | 15 +- include/linux/skbuff.h | 4 +- include/net/sch_generic.h | 2 +- include/net/tcx.h | 199 ++ include/uapi/linux/bpf.h | 70 +- kernel/bpf/Kconfig | 1 + kernel/bpf/Makefile | 3 +- kernel/bpf/mprog.c | 427 ++++ kernel/bpf/syscall.c | 83 +- kernel/bpf/tcx.c | 351 +++ net/Kconfig | 5 + net/core/dev.c | 267 +- net/core/filter.c | 4 +- net/sched/Kconfig | 4 +- net/sched/sch_ingress.c | 61 +- tools/bpf/bpftool/net.c | 86 +- tools/include/uapi/linux/bpf.h | 70 +- tools/lib/bpf/bpf.c | 124 +- tools/lib/bpf/bpf.h | 97 +- tools/lib/bpf/libbpf.c | 74 +- tools/lib/bpf/libbpf.h | 16 + tools/lib/bpf/libbpf.map | 2 + tools/lib/bpf/libbpf_common.h | 11 + .../selftests/bpf/prog_tests/tc_helpers.h | 72 + .../selftests/bpf/prog_tests/tc_links.c | 1604 ++++++++++++ .../selftests/bpf/prog_tests/tc_opts.c | 2182 +++++++++++++++++ .../selftests/bpf/progs/test_tc_link.c | 40 + 29 files changed, 6002 insertions(+), 229 deletions(-) create mode 100644 include/linux/bpf_mprog.h create mode 100644 include/net/tcx.h create mode 100644 kernel/bpf/mprog.c create mode 100644 kernel/bpf/tcx.c create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_helpers.h create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_links.c create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_opts.c create mode 100644 tools/testing/selftests/bpf/progs/test_tc_link.c
Signature · 1 line
2.34.1
Daniel Borkmann
maintainerPATCH v4 1/8 bpf: Add generic attach/detach/query API for multi-progs
This adds a generic layer called bpf_mprog which can be reused by different attachment layers to enable multi-program attachment and dependency resolution. In-kernel users of the bpf_mprog don't need to care about the dependency resolution internals, they can just consume it with few API calls.
The initial idea of having a generic API sparked out of discussion [0] from an earlier revision of this work where tc's priority was reused and exposed via BPF uapi as a way to coordinate dependencies among tc BPF programs, similar as-is for classic tc BPF. The feedback was that priority provides a bad user experience and is hard to use [1], e.g.:
I cannot help but feel that priority logic copy-paste from old tc, netfilter and friends is done because "that's how things were done in the past". [...] Priority gets exposed everywhere in uapi all the way to bpftool when it's right there for users to understand. And that's the main problem with it.
The user don't want to and don't need to be aware of it, but uapi forces them to pick the priority. [...] Your cover letter [0] example proves that in real life different service pick the same priority. They simply don't know any better. Priority is an unnecessary magic that apps _have_ to pick, so they just copy-paste and everyone ends up using the same.
The course of the discussion showed more and more the need for a generic, reusable API where the "same look and feel" can be applied for various other program types beyond just tc BPF, for example XDP today does not have multi- program support in kernel, but also there was interest around this API for improving management of cgroup program types. Such common multi-program management concept is useful for BPF management daemons or user space BPF applications coordinating internally about their attachments.
Both from Cilium and Meta side [2], we've collected the following requirements for a generic attach/detach/query API for multi-progs which has been implemented as part of this work:
- Support prog-based attach/detach and link API
- Dependency directives (can also be combined):
- BPF_F_{BEFORE,AFTER} with relative_{fd,id} which can be {prog,link,none}
- BPF_F_ID flag as {fd,id} toggle; the rationale for id is so that user space application does not need CAP_SYS_ADMIN to retrieve foreign fds via bpf_*_get_fd_by_id()
- BPF_F_LINK flag as {prog,link} toggle
- If relative_{fd,id} is none, then BPF_F_BEFORE will just prepend, and BPF_F_AFTER will just append for attaching
- Enforced only at attach time
- BPF_F_REPLACE with replace_bpf_fd which can be prog, links have their own infra for replacing their internal prog
- If no flags are set, then it's default append behavior for attaching
- Internal revision counter and optionally being able to pass expected_revision
- User space application can query current state with revision, and pass it along for attachment to assert current state before doing updates
- Query also gets extension for link_ids array and link_attach_flags:
- prog_ids are always filled with program IDs
- link_ids are filled with link IDs when link was used, otherwise 0
- {prog,link}_attach_flags for holding {prog,link}-specific flags
- Must be easy to integrate/reuse for in-kernel users
The uapi-side changes needed for supporting bpf_mprog are rather minimal, consisting of the additions of the attachment flags, revision counter, and expanding existing union with relative_{fd,id} member.
The bpf_mprog framework consists of an bpf_mprog_entry object which holds an array of bpf_mprog_fp (fast-path structure). The bpf_mprog_cp (control-path structure) is part of bpf_mprog_bundle. Both have been separated, so that fast-path gets efficient packing of bpf_prog pointers for maximum cache efficiency. Also, array has been chosen instead of linked list or other structures to remove unnecessary indirections for a fast point-to-entry in tc for BPF.
The bpf_mprog_entry comes as a pair via bpf_mprog_bundle so that in case of updates the peer bpf_mprog_entry is populated and then just swapped which avoids additional allocations that could otherwise fail, for example, in detach case. bpf_mprog_{fp,cp} arrays are currently static, but they could be converted to dynamic allocation if necessary at a point in future. Locking is deferred to the in-kernel user of bpf_mprog, for example, in case of tcx which uses this API in the next patch, it piggybacks on rtnl.
An extensive test suite for checking all aspects of this API for prog-based attach/detach and link API comes as BPF selftests in this series.
Kudos also to Andrii Nakryiko for API discussions wrt Meta's BPF management.
[0] https://lore.kernel.org/bpf/[email protected]
[1] https://lore.kernel.org/bpf/CAADnVQ+gEY3FjCR=+DmjDR4gp5bOYZUFJQXj4agKFHT9CQPZBw@mail.gmail.com
[2] http://vger.kernel.org/bpfconf2023_material/tcx_meta_netdev_borkmann.pdf
--- MAINTAINERS | 1 + include/linux/bpf_mprog.h | 343 ++++++++++++++++++++++++++ include/uapi/linux/bpf.h | 36 ++- kernel/bpf/Makefile | 2 +- kernel/bpf/mprog.c | 427 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 36 ++- 6 files changed, 828 insertions(+), 17 deletions(-) create mode 100644 include/linux/bpf_mprog.h create mode 100644 kernel/bpf/mprog.c
MAINTAINERS +1 -0
@@ -3736,6 +3736,7 @@ F: include/linux/filter.h F: include/linux/tnum.h F: kernel/bpf/core.c F: kernel/bpf/dispatcher.c +F: kernel/bpf/mprog.c F: kernel/bpf/syscall.c F: kernel/bpf/tnum.c F: kernel/bpf/trampoline.c
include/linux/bpf_mprog.h new file +343 -0
343 lines not on this page · raw message
include/uapi/linux/bpf.h +28 -8
@@ -1103,7 +1103,12 @@ enum bpf_link_type {*/ #define BPF_F_ALLOW_OVERRIDE (1U << 0)#define BPF_F_ALLOW_MULTI (1U << 1) +/* Generic attachment flags. */ #define BPF_F_REPLACE (1U << 2) +#define BPF_F_BEFORE (1U << 3) +#define BPF_F_AFTER (1U << 4) +#define BPF_F_ID (1U << 5) +#define BPF_F_LINK BPF_F_LINK/* 1 << 13 */ /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the * verifier will perform strict alignment checking as if the kernel
@@ -1434,14 +1439,19 @@ union bpf_attr { };struct {/* anonymous struct used by BPF_PROG_ATTACH/DETACH commands */ -__u32 target_fd;/* container object to attach to */ -__u32 attach_bpf_fd;/* eBPF program to attach */ +union { +__u32 target_fd;/* target object to attach to or ... */ +__u32 target_ifindex;/* target ifindex */ + }; +__u32 attach_bpf_fd;__u32 attach_type;__u32 attach_flags; -__u32 replace_bpf_fd;/* previously attached eBPF -* program to replace if -* BPF_F_REPLACE is used -*/ +__u32 replace_bpf_fd; +union { +__u32 relative_fd; +__u32 relative_id; + }; +__u64 expected_revision; };struct {/* anonymous struct used by BPF_PROG_TEST_RUN command */
@@ -1487,16 +1497,26 @@ union bpf_attr { } info;struct {/* anonymous struct used by BPF_PROG_QUERY command */ -__u32 target_fd;/* container object to query */ +union { +__u32 target_fd;/* target object to query or ... */ +__u32 target_ifindex;/* target ifindex */ + };__u32 attach_type;__u32 query_flags;__u32 attach_flags;__aligned_u64 prog_ids; -__u32 prog_cnt; +union { +__u32 prog_cnt; +__u32 count; + }; + __u32 :32;/* output: per-program attach_flags. * not allowed to be set during effective query. */ __aligned_u64 prog_attach_flags; +__aligned_u64 link_ids; +__aligned_u64 link_attach_flags; +__u64 revision; } query;struct {/* anonymous struct used by BPF_RAW_TRACEPOINT_OPEN command */
kernel/bpf/Makefile +1 -1
@@ -12,7 +12,7 @@ obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o -obj-$(CONFIG_BPF_SYSCALL) += disasm.o +obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o obj-$(CONFIG_BPF_JIT) += trampoline.o obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o obj-$(CONFIG_BPF_JIT) += dispatcher.o
kernel/bpf/mprog.c new file +427 -0
427 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +28 -8
@@ -1103,7 +1103,12 @@ enum bpf_link_type {*/ #define BPF_F_ALLOW_OVERRIDE (1U << 0)#define BPF_F_ALLOW_MULTI (1U << 1) +/* Generic attachment flags. */ #define BPF_F_REPLACE (1U << 2) +#define BPF_F_BEFORE (1U << 3) +#define BPF_F_AFTER (1U << 4) +#define BPF_F_ID (1U << 5) +#define BPF_F_LINK BPF_F_LINK/* 1 << 13 */ /* If BPF_F_STRICT_ALIGNMENT is used in BPF_PROG_LOAD command, the * verifier will perform strict alignment checking as if the kernel
@@ -1434,14 +1439,19 @@ union bpf_attr { };struct {/* anonymous struct used by BPF_PROG_ATTACH/DETACH commands */ -__u32 target_fd;/* container object to attach to */ -__u32 attach_bpf_fd;/* eBPF program to attach */ +union { +__u32 target_fd;/* target object to attach to or ... */ +__u32 target_ifindex;/* target ifindex */ + }; +__u32 attach_bpf_fd;__u32 attach_type;__u32 attach_flags; -__u32 replace_bpf_fd;/* previously attached eBPF -* program to replace if -* BPF_F_REPLACE is used -*/ +__u32 replace_bpf_fd; +union { +__u32 relative_fd; +__u32 relative_id; + }; +__u64 expected_revision; };struct {/* anonymous struct used by BPF_PROG_TEST_RUN command */
@@ -1487,16 +1497,26 @@ union bpf_attr { } info;struct {/* anonymous struct used by BPF_PROG_QUERY command */ -__u32 target_fd;/* container object to query */ +union { +__u32 target_fd;/* target object to query or ... */ +__u32 target_ifindex;/* target ifindex */ + };__u32 attach_type;__u32 query_flags;__u32 attach_flags;__aligned_u64 prog_ids; -__u32 prog_cnt; +union { +__u32 prog_cnt; +__u32 count; + }; + __u32 :32;/* output: per-program attach_flags. * not allowed to be set during effective query. */ __aligned_u64 prog_attach_flags; +__aligned_u64 link_ids; +__aligned_u64 link_attach_flags; +__u64 revision; } query;struct {/* anonymous struct used by BPF_RAW_TRACEPOINT_OPEN command */
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v4 1/8] bpf: Add generic attach/detach/query API for multi-progs
Quoted 33 lines · Daniel Borkmann
+ *
+ * structbpf_mprog_entry*entry, *peer;
+ * int ret;
+ *+ * // bpf_mprog user-side lock + * // fetch active @entry from attach location + * [...] + * ret = bpf_mprog_attach(entry, [...]); + * if (ret >= 0) {+ * peer =
bpf_mprog_peer(entry);
+ * if (bpf_mprog_swap_entries(ret))
+ * // swap @entry to @peer at attach location
+ *bpf_mprog_commit(entry);
+ * ret = 0;
+ * } else {
+ * // error path, bail out, propagate @ret
+ * }
+ * //bpf_mproguser-side unlock
+ *
+ * Detach case:
+ *+ * struct bpf_mprog_entry *entry, *peer; + * bool release; + * int ret; + * + * // bpf_mprog user-side lock + * // fetch active @entry from attach location + * [...] + * ret = bpf_mprog_detach(entry, [...]); + * if (ret >= 0) { + * release = ret == BPF_MPROG_FREE; + * peer = release ? NULL : bpf_mprog_peer(entry); + * if (bpf_mprog_swap_entries(ret))
Quoted 9 lines
+ * // swap @entry to @peer at attach location
+ *bpf_mprog_commit(entry);
+ * if (release)
+ * // freebpf_mprog_bundle
+ * ret = 0;
+ * } else {
+ * // error path, bail out, propagate @ret
+ * }
+ * //bpf_mproguser-side unlock
Thanks for the doc. It helped a lot.
And when it's contained like this it's easier to discuss api. It seems bpf_mprog_swap_entries() is trying to abstract the error code away, but BPF_MPROG_FREE leaks out and tcx_entry_needs_release() captures it with extra miniq_active twist, which I don't understand yet. bpf_mprog_peer() is also leaking a bit of implementation detail. Can we abstract it further, like:
ret = bpf_mprog_detach(entry, [...], &new_entry);
if (ret >= 0) {
if (entry != new_entry)
// swap @entry to @new_entry at attach location
bpf_mprog_commit(entry);
if (!new_entry)
// free bpf_mprog_bundle
}
and make bpf_mprog_peer internal to mprog. It will also allow removing
BPF_MPROG_FREE vs SWAP distinction. peer is hidden.
if (entry != new_entry)
// updatealso will be easier to read inside tcx code without looking into mprog details.
Re: [PATCH bpf-next v4 1/8] bpf: Add generic attach/detach/query API for multi-progs
Quoted 65 lines · Alexei Starovoitov
Quoted 33 lines · Daniel Borkmann
+ *
+ * structbpf_mprog_entry*entry, *peer;
+ * int ret;
+ *+ * // bpf_mprog user-side lock + * // fetch active @entry from attach location + * [...] + * ret = bpf_mprog_attach(entry, [...]); + * if (ret >= 0) {+ * peer =
bpf_mprog_peer(entry);
+ * if (bpf_mprog_swap_entries(ret))
+ * // swap @entry to @peer at attach location
+ *bpf_mprog_commit(entry);
+ * ret = 0;
+ * } else {
+ * // error path, bail out, propagate @ret
+ * }
+ * //bpf_mproguser-side unlock
+ *
+ * Detach case:
+ *+ * struct bpf_mprog_entry *entry, *peer; + * bool release; + * int ret; + * + * // bpf_mprog user-side lock + * // fetch active @entry from attach location + * [...] + * ret = bpf_mprog_detach(entry, [...]); + * if (ret >= 0) { + * release = ret == BPF_MPROG_FREE; + * peer = release ? NULL : bpf_mprog_peer(entry); + * if (bpf_mprog_swap_entries(ret))Quoted 9 lines
+ * // swap @entry to @peer at attach location
+ *bpf_mprog_commit(entry);
+ * if (release)
+ * // freebpf_mprog_bundle
+ * ret = 0;
+ * } else {
+ * // error path, bail out, propagate @ret
+ * }
+ * //bpf_mproguser-side unlockThanks for the doc. It helped a lot.
And when it's contained like this it's easier to discuss api. It seemsbpf_mprog_swap_entries()is trying to abstract the error code away, butBPF_MPROG_FREEleaks out andtcx_entry_needs_release()captures it with extraminiq_activetwist, which I don't understand yet.bpf_mprog_peer()is also leaking a bit of implementation detail. Can we abstract it further, like:ret =
bpf_mprog_detach(entry, [...], &new_entry);if (ret >= 0) { if (entry != new_entry) // swap @entry to @new_entry at attach location bpf_mprog_commit(entry); if (!new_entry) // free bpf_mprog_bundle } and make bpf_mprog_peer internal to mprog. It will also allow removing BPF_MPROG_FREE vs SWAP distinction. peer is hidden. if (entry != new_entry) // update
Quoted 1 line
also will be easier to read inside tcx code without looking into mprog details.
I'm actually thinking if it's possible to simplify it even further. For example, do we even need a separate bpf_mprog_{attach,detach} and bpf_mprog_commit()? So far it seems like bpf_mprog_commit() is inevitable in case of success of attach/detach, so we might as well just do it as the last step of attach/detach operation.
The only problem seems to be due to bpf_mprog interface doing this optimization of replacing stuff in place, if possible, and allowing the caller to not do the swap. How important is it to avoid that swap of a bpf_mprog_fp (pointer)? Seems pretty cheap (and relatively rare operation), so I wouldn't bother optimizing this.
So how about we just say that there is always a swap. Internally in bpf_mprog_bundle current entry is determined based on revision&1. We can have bpf_mprog_cur_entry() to return a proper pointer after commit. Or bpf_mprog_attach() can return proper new entry as output parameter, whichever is preferable.
As for BPF_MPROG_FREE. That seems like an unnecessary complication as well. Caller can just check bpf_mprog_total() quickly, and if it dropped to zero assume FREE. Unless there is something more subtle there?
With the above, the interface will be much simpler, IMO. You just do bpf_mprog_attach/detach, and then swap pointer to new bpf_mprog_entry. Then you can check bpf_mprog_total() for zero, and clean up further, if necessary.
We assume the caller has a proper locking, so all the above should be non-racy.
BTW, combining commit with attach allows us to avoid that relatively big bpf_mprog_cp array on the stack as well, because we will be able to update bundle->cp_items in-place.
The only (I believe :) ) big assumption I'm making in all of the above is that commit is inevitable and we won't have a situation where we start attach, update fp/cpp, and then decide to abort instead of going for commit. Is this possible? Can we avoid it by careful checks upfront and doing attach as last step that cannot be undone?
P.S. I guess one bit that I might have simplified is that synchronize_rcu() + bpf_prog_put(), but I'm not sure exactly why we put prog after sync_rcu. But if it's really necessary (and I assume it is) and is a blocker for the proposal above, then maybe the interface should delegate that to the caller (i.e., optionally return replaced prog pointer from attach/detach) or use call_rcu() with callback?
Daniel Borkmann
maintainerRe: [PATCH bpf-next v4 1/8] bpf: Add generic attach/detach/query API for multi-progs
Quoted 55 lines · Andrii Nakryiko
Quoted 53 lines · Alexei Starovoitov
Quoted 33 lines · Daniel Borkmann
+ *
+ * structbpf_mprog_entry*entry, *peer;
+ * int ret;
+ *+ * // bpf_mprog user-side lock + * // fetch active @entry from attach location + * [...] + * ret = bpf_mprog_attach(entry, [...]); + * if (ret >= 0) {+ * peer =
bpf_mprog_peer(entry);
+ * if (bpf_mprog_swap_entries(ret))
+ * // swap @entry to @peer at attach location
+ *bpf_mprog_commit(entry);
+ * ret = 0;
+ * } else {
+ * // error path, bail out, propagate @ret
+ * }
+ * //bpf_mproguser-side unlock
+ *
+ * Detach case:
+ *+ * struct bpf_mprog_entry *entry, *peer; + * bool release; + * int ret; + * + * // bpf_mprog user-side lock + * // fetch active @entry from attach location + * [...] + * ret = bpf_mprog_detach(entry, [...]); + * if (ret >= 0) { + * release = ret == BPF_MPROG_FREE; + * peer = release ? NULL : bpf_mprog_peer(entry); + * if (bpf_mprog_swap_entries(ret))Quoted 9 lines
+ * // swap @entry to @peer at attach location
+ *bpf_mprog_commit(entry);
+ * if (release)
+ * // freebpf_mprog_bundle
+ * ret = 0;
+ * } else {
+ * // error path, bail out, propagate @ret
+ * }
+ * //bpf_mproguser-side unlockThanks for the doc. It helped a lot.
And when it's contained like this it's easier to discuss api. It seemsbpf_mprog_swap_entries()is trying to abstract the error code away, butBPF_MPROG_FREEleaks out andtcx_entry_needs_release()captures it with extraminiq_activetwist, which I don't understand yet.bpf_mprog_peer()is also leaking a bit of implementation detail. Can we abstract it further, like:
Quoted 13 lines
Quoted 12 lines
ret =
bpf_mprog_detach(entry, [...], &new_entry);if (ret >= 0) { if (entry != new_entry) // swap @entry to @new_entry at attach location bpf_mprog_commit(entry); if (!new_entry) // free bpf_mprog_bundle } and make bpf_mprog_peer internal to mprog. It will also allow removing BPF_MPROG_FREE vs SWAP distinction. peer is hidden. if (entry != new_entry) // updateQuoted 1 line
also will be easier to read inside tcx code without looking into mprog details.
+1, agree, and I implemented this suggestion in the v5.
Quoted 5 lines
I'm actually thinking if it's possible to simplify it even further. For example, do we even need a separate bpf_mprog_{attach,detach} and
bpf_mprog_commit()? So far it seems likebpf_mprog_commit()is inevitable in case of success of attach/detach, so we might as well just do it as the last step of attach/detach operation.
It needs to be done after the pointers have been swapped by the mprog user.
Quoted 5 lines
The only problem seems to be due to
bpf_mproginterface doing this optimization of replacing stuff in place, if possible, and allowing the caller to not do the swap. How important is it to avoid that swap of abpf_mprog_fp(pointer)? Seems pretty cheap (and relatively rare operation), so I wouldn't bother optimizing this.
I would like to keep it given e.g. when application comes up, fetches links from bpffs and updates all its programs in place, then this is the replace situation.
Quoted 10 lines
So how about we just say that there is always a swap. Internally in
bpf_mprog_bundlecurrent entry is determined based on revision&1. We can havebpf_mprog_cur_entry()to return a proper pointer after commit. Orbpf_mprog_attach()can return proper new entry as output parameter, whichever is preferable.As for
BPF_MPROG_FREE. That seems like an unnecessary complication as well. Caller can just checkbpf_mprog_total()quickly, and if it dropped to zero assume FREE. Unless there is something more subtle there?
Agree, some may want to keep an empty bpf_mprog, others may want to free it. I implemented it this way. I removed all the BPF_MPROG_* return codes.
Quoted 17 lines
With the above, the interface will be much simpler, IMO. You just do bpf_mprog_attach/detach, and then swap pointer to new
bpf_mprog_entry. Then you can checkbpf_mprog_total()for zero, and clean up further, if necessary.We assume the caller has a proper locking, so all the above should be non-racy.
BTW, combining commit with attach allows us to avoid that relatively big
bpf_mprog_cparray on the stack as well, because we will be able to updatebundle->cp_itemsin-place.The only (I believe :) ) big assumption I'm making in all of the above is that commit is inevitable and we won't have a situation where we start attach, update fp/cpp, and then decide to abort instead of going for commit. Is this possible? Can we avoid it by careful checks upfront and doing attach as last step that cannot be undone?
Quoted 3 lines
P.S. I guess one bit that I might have simplified is thatsynchronize_rcu()+bpf_prog_put(), but I'm not sure exactly why we put prog aftersync_rcu. But if it's really necessary (and I assume it
It is because users can still be inflight on the old mprog_entry so it must come after the sync rcu where we drop ref for the delete case.
Thanks again,
Daniel
Re: [PATCH bpf-next v4 1/8] bpf: Add generic attach/detach/query API for multi-progs
In reply to Daniel BorkmannQuoted 87 lines · Daniel Borkmann
This adds a generic layer called
bpf_mprogwhich can be reused by different attachment layers to enable multi-program attachment and dependency resolution. In-kernel users of thebpf_mprogdon't need to care about the dependency resolution internals, they can just consume it with few API calls.The initial idea of having a generic API sparked out of discussion [0] from an earlier revision of this work where tc's priority was reused and exposed via BPF uapi as a way to coordinate dependencies among tc BPF programs, similar as-is for classic tc BPF. The feedback was that priority provides a bad user experience and is hard to use [1], e.g.:
I cannot help but feel that priority logic copy-paste from old tc, netfilter and friends is done because "that's how things were done in the past". [...] Priority gets exposed everywhere in uapi all the way to bpftool when it's right there for users to understand. And that's the main problem with it.
The user don't want to and don't need to be aware of it, but uapi forces them to pick the priority. [...] Your cover letter [0] example proves that in real life different service pick the same priority. They simply don't know any better. Priority is an unnecessary magic that apps _have_ to pick, so they just copy-paste and everyone ends up using the same.
The course of the discussion showed more and more the need for a generic, reusable API where the "same look and feel" can be applied for various other program types beyond just tc BPF, for example XDP today does not have multi- program support in kernel, but also there was interest around this API for improving management of cgroup program types. Such common multi-program management concept is useful for BPF management daemons or user space BPF applications coordinating internally about their attachments.
Both from Cilium and Meta side [2], we've collected the following requirements for a generic attach/detach/query API for multi-progs which has been implemented as part of this work:
- Support prog-based attach/detach and link API
- Dependency directives (can also be combined):
- BPF_F_{BEFORE,AFTER} with relative_{fd,id} which can be {prog,link,none}
-BPF_F_IDflag as {fd,id} toggle; the rationale for id is so that user space application does not needCAP_SYS_ADMINto retrieve foreign fds via bpf_*_get_fd_by_id()
-BPF_F_LINKflag as {prog,link} toggle
- If relative_{fd,id} is none, thenBPF_F_BEFOREwill just prepend, andBPF_F_AFTERwill just append for attaching
- Enforced only at attach time
-BPF_F_REPLACEwithreplace_bpf_fdwhich can be prog, links have their own infra for replacing their internal prog
- If no flags are set, then it's default append behavior for attaching
- Internal revision counter and optionally being able to passexpected_revision
- User space application can query current state with revision, and pass it along for attachment to assert current state before doing updates
- Query also gets extension forlink_idsarray andlink_attach_flags:
-prog_idsare always filled with program IDs
-link_idsare filled with link IDs when link was used, otherwise 0
- {prog,link}_attach_flags for holding {prog,link}-specific flags
- Must be easy to integrate/reuse for in-kernel usersThe uapi-side changes needed for supporting
bpf_mprogare rather minimal, consisting of the additions of the attachment flags, revision counter, and expanding existing union with relative_{fd,id} member.The
bpf_mprogframework consists of anbpf_mprog_entryobject which holds an array ofbpf_mprog_fp(fast-path structure). Thebpf_mprog_cp(control-path structure) is part ofbpf_mprog_bundle. Both have been separated, so that fast-path gets efficient packing ofbpf_progpointers for maximum cache efficiency. Also, array has been chosen instead of linked list or other structures to remove unnecessary indirections for a fast point-to-entry in tc for BPF.The
bpf_mprog_entrycomes as a pair viabpf_mprog_bundleso that in case of updates the peerbpf_mprog_entryis populated and then just swapped which avoids additional allocations that could otherwise fail, for example, in detach case. bpf_mprog_{fp,cp} arrays are currently static, but they could be converted to dynamic allocation if necessary at a point in future. Locking is deferred to the in-kernel user ofbpf_mprog, for example, in case of tcx which uses this API in the next patch, it piggybacks on rtnl.An extensive test suite for checking all aspects of this API for prog-based attach/detach and link API comes as BPF selftests in this series.
Kudos also to Andrii Nakryiko for API discussions wrt Meta's BPF management.
[0] https://lore.kernel.org/bpf/[email protected]
[1] https://lore.kernel.org/bpf/CAADnVQ+gEY3FjCR=+DmjDR4gp5bOYZUFJQXj4agKFHT9CQPZBw@mail.gmail.com
[2] http://vger.kernel.org/bpfconf2023_material/tcx_meta_netdev_borkmann.pdfSigned-off-by Daniel Borkmann <[email protected]>
Quoted 11 lines
--- MAINTAINERS | 1 + include/linux/bpf_mprog.h | 343 ++++++++++++++++++++++++++ include/uapi/linux/bpf.h | 36 ++- kernel/bpf/Makefile | 2 +- kernel/bpf/mprog.c | 427 +++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 36 ++- 6 files changed, 828 insertions(+), 17 deletions(-) create mode 100644 include/linux/bpf_mprog.h create mode 100644 kernel/bpf/mprog.c
From UAPI perspective looks great! Few implementation suggestion below. I'll also reply separately to Alexei's reply with discussion on higher-level *internal* API.
[...]
Quoted 5 lines
include/linux/bpf_mprog.h new file +5
-0+ +#define BPF_MPROG_KEEP 0 +#define BPF_MPROG_SWAP 1 +#define BPF_MPROG_FREE 2 +
include/linux/bpf_mprog.h line 146
Quoted 10 lines
include/linux/bpf_mprog.h new file +10
-0+#define BPF_MPROG_MAX 64 + +#define bpf_mprog_foreach_tuple (entry, fp, cp, t) \ + for (fp = &entry->fp_items[0], cp = &entry->parent->cp_items[0];\ + ({ \ + t.prog = READ_ONCE(fp->prog); \ + t.link = cp->link; \ + t.prog; \ + }); \ + fp++, cp++)
I wish we could do something like the below to avoid the need to pass fp and cp from outside:
for (struct { struct bpf_mprog_fp *fp; struct bpf_mprog_cp *cp;} tmp =
{ &entry->fp_items[0], &entry->parent->cp_iterms[0]};
t.link = tmp.cp->link, t.prog = READ_ONCE(tmp.fp->prog);
fp++, cp++)But I'm not sure the kernel's C style allows that yet.
But I think you can use the comma operator to avoid that more verbose ({ }) construct.
Quoted 6 lines
include/linux/bpf_mprog.h new file +6
-0+ +#define bpf_mprog_foreach_prog (entry, fp, p) \ + for (fp = &entry->fp_items[0]; \ + (p = READ_ONCE(fp->prog)); \ + fp++) +
[...]
Quoted 4 lines
include/linux/bpf_mprog.h new file +4
-0+static inline void bpf_mprog_inc (struct bpf_mprog_entry *entry) +{ + entry->parent->count++; +}
include/linux/bpf_mprog.h line 214
Quoted 10 lines
include/linux/bpf_mprog.h new file +10
-0+ +static inline void bpf_mprog_dec (struct bpf_mprog_entry *entry) +{ + entry->parent->count--; +} + +static inline int bpf_mprog_max (void ) +{ +return ARRAY_SIZE (((struct bpf_mprog_entry *)NULL )->fp_items) -1 ; +}
so we can only add BPF_MPROG_MAX - 1 progs, right? I presume the last entry is presumed to be always NULL, right?
Quoted 9 lines
include/linux/bpf_mprog.h new file +9
-0+ +static inline int bpf_mprog_total (struct bpf_mprog_entry *entry) +{ +int total = entry->parent->count; + +WARN_ON_ONCE (total >bpf_mprog_max ()); +return total; +} +
[...]
Quoted 19 lines
kernel/bpf/Makefile +1
-1@@ -12,7 +12,7 @@ obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o -obj-$(CONFIG_BPF_SYSCALL) += disasm.o +obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o obj-$(CONFIG_BPF_JIT) += trampoline.o obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o obj-$(CONFIG_BPF_JIT) += dispatcher.okernel/bpf/mprog.c new file +0
-0
kernel/bpf/mprog.c line 8
Quoted 8 lines
kernel/bpf/mprog.c new file +8
-0@@ -0,0 +1,427 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Isovalent */ + +#include <linux/bpf.h> +#include <linux/bpf_mprog.h> + +static int bpf_mprog_link(struct bpf_tuple *tuple, + u32 object, u32 flags,
so I tried to get used to this "object" notation, but I think it's still awkwards and keeps me asking "what is this really" every single time I read this. I wonder if something like "fd_or_id" as a name would make it more obvious?
kernel/bpf/mprog.c line 13
Quoted 5 lines
kernel/bpf/mprog.c new file +5
-0+ enum bpf_prog_type type) +{ +bool id = flags &BPF_F_ID ; +struct bpf_link *link; +
should we reject this object/fd_or_id if it's zero, instead of trying to lookup ID/FD 0?
Quoted 13 lines
kernel/bpf/mprog.c new file +13
-0+if (id) + link =bpf_link_by_id (object); +else + link =bpf_link_get_from_fd (object); +if (IS_ERR (link)) +return PTR_ERR (link); +if (type && link->prog->type != type) { +bpf_link_put (link); +return -EINVAL ; + } + + tuple->link = link; + tuple->prog = link->prog;
kernel/bpf/mprog.c line 36
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+return 0 ; +} + +static int bpf_mprog_prog (struct bpf_tuple *tuple, +u32 object,u32 flags, +enum bpf_prog_type type) +{ +bool id = flags &BPF_F_ID ; +struct bpf_prog *prog; +
same here about rejecting zero object?
Quoted 49 lines
kernel/bpf/mprog.c new file +49
-0+if (id) + prog =bpf_prog_by_id (object); +else + prog =bpf_prog_get (object); +if (IS_ERR (prog)) { +if (!object && !id) +return 0 ; +return PTR_ERR (prog); + } +if (type && prog->type != type) { +bpf_prog_put (prog); +return -EINVAL ; + } + + tuple->link =NULL ; + tuple->prog = prog; +return 0 ; +} + +static int bpf_mprog_tuple_relative (struct bpf_tuple *tuple, +u32 object,u32 flags, +enum bpf_prog_type type) +{ +memset (tuple,0 ,sizeof (*tuple)); +if (flags &BPF_F_LINK ) +return bpf_mprog_link (tuple, object, flags, type); +return bpf_mprog_prog (tuple, object, flags, type); +} + +static void bpf_mprog_tuple_put (struct bpf_tuple *tuple) +{ +if (tuple->link) +bpf_link_put (tuple->link); +else if (tuple->prog) +bpf_prog_put (tuple->prog); +} + +static int bpf_mprog_replace (struct bpf_mprog_entry *entry, +struct bpf_tuple *ntuple,int idx) +{ +struct bpf_mprog_fp *fp; +struct bpf_mprog_cp *cp; +struct bpf_prog *oprog; + +bpf_mprog_read (entry, idx, &fp, &cp); + oprog =READ_ONCE (fp->prog); +bpf_mprog_write (fp, cp, ntuple); +if (!ntuple->link) { +WARN_ON_ONCE (cp->link);
kernel/bpf/mprog.c line 95
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+bpf_prog_put (oprog); + } +return BPF_MPROG_KEEP ; +} + +static int bpf_mprog_insert (struct bpf_mprog_entry *entry, +struct bpf_tuple *ntuple,int idx,u32 flags) +{ +int i, j =0 , total =bpf_mprog_total (entry); +struct bpf_mprog_cp *cp, cpp[BPF_MPROG_MAX ] = {};
a bit worried about using 512 bytes for local cpp array... my initial assumption was that we won't have to create a copy of cp_iterms, just update it in place. Hm... let's have the higher-level API discussion in one branch, where Alexei has some proposals as well.
Quoted 19 lines
kernel/bpf/mprog.c new file +19
-0+struct bpf_mprog_fp *fp, *fpp; +struct bpf_mprog_entry *peer; + + peer =bpf_mprog_peer (entry); +bpf_mprog_entry_clear (peer); +if (idx <0 ) { +bpf_mprog_read_fp (peer, j, &fpp); +bpf_mprog_write_fp (fpp, ntuple); +bpf_mprog_write_cp (&cpp[j], ntuple); + j++; + } +for (i =0 ; i <= total; i++) { +bpf_mprog_read_fp (peer, j, &fpp); +if (idx == i && (flags &BPF_F_AFTER )) { +bpf_mprog_write (fpp, &cpp[j], ntuple); + j++; +bpf_mprog_read_fp (peer, j, &fpp); + } +if (i < total) {
kernel/bpf/mprog.c line 124
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+bpf_mprog_read (entry, i, &fp, &cp); +bpf_mprog_copy (fpp, &cpp[j], fp, cp); + j++; + } +if (idx == i && (flags &BPF_F_BEFORE )) { +bpf_mprog_read_fp (peer, j, &fpp); +bpf_mprog_write (fpp, &cpp[j], ntuple); + j++; + } + }
sorry if I miss some subtle point, but I wonder why this is so complicated? I think this choice of idx == -1 meaning prepend is leading to this complication. It's not also clear why there is this BPF_F_AFTER vs BPF_F_BEFORE distinction when we already determined a position where new program has to be inserted (so after or before should be irrelevant).
Please let me know why the below doesn't work.
Let's define that idx is the position where new prog/link tuple has to be inserted. It can be in the range [0, N], where N is number of programs currently in the mprog_peer. Note that N is inclusive above.
The algorithm for insertion is simple: everything currently at entry->fp_items[idx] and after gets shifted. And we can do it with a simple memmove:
memmove(peer->fp_items + idx + 1, peer->fp_iters + idx, (bpf_mprog_total(entry) - idx) * sizeof(struct bpf_mprof_fp)); /* similar memmove for cp_items/cpp array, of course */ /* now set new prog at peer->fp_items[idx] */
The above should replace entire above for loop and that extra if before the loop. And it should work for corner cases:
- idx == 0 (prepend), will shift everything to the right, and put
new prog at position 0. Exactly what we wanted.
- idx == N (append), will shift nothing (that memmov should be a
no-op because size is zero, total == idx == N)
We just need to make sure that the above shift won't overwrite the very last NULL. So bpf_mprog_total() should be < BPF_MPROG_MAX - 2 before all this.
Seems as simple as that, is there any complication I skimmed over?
Quoted 54 lines
kernel/bpf/mprog.c new file +54
-0+bpf_mprog_commit_cp (peer, cpp); +bpf_mprog_inc (peer); +return BPF_MPROG_SWAP ; +} + +static int bpf_mprog_tuple_confirm (struct bpf_mprog_entry *entry, +struct bpf_tuple *dtuple,int idx) +{ +int first =0 , last =bpf_mprog_total (entry) -1 ; +struct bpf_mprog_cp *cp; +struct bpf_mprog_fp *fp; +struct bpf_prog *prog; +struct bpf_link *link; + +if (idx <= first) +bpf_mprog_read (entry, first, &fp, &cp); +else if (idx >= last) +bpf_mprog_read (entry, last, &fp, &cp); +else +bpf_mprog_read (entry, idx, &fp, &cp); + + prog =READ_ONCE (fp->prog); + link = cp->link; +if (!dtuple->link && link) +return -EBUSY ; + +WARN_ON_ONCE (dtuple->prog && dtuple->prog != prog); +WARN_ON_ONCE (dtuple->link && dtuple->link != link); + + dtuple->prog = prog; + dtuple->link = link; +return 0 ; +} + +static int bpf_mprog_delete (struct bpf_mprog_entry *entry, +struct bpf_tuple *dtuple,int idx) +{ +int i =0 , j, ret, total =bpf_mprog_total (entry); +struct bpf_mprog_cp *cp, cpp[BPF_MPROG_MAX ] = {}; +struct bpf_mprog_fp *fp, *fpp; +struct bpf_mprog_entry *peer; + + ret =bpf_mprog_tuple_confirm (entry, dtuple, idx); +if (ret) +return ret; + peer =bpf_mprog_peer (entry); +bpf_mprog_entry_clear (peer); +if (idx <0 ) + i++; +if (idx == total) + total--; +for (j =0 ; i < total; i++) { +if (idx == i) +continue ;
kernel/bpf/mprog.c line 188
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+bpf_mprog_read_fp (peer, j, &fpp); +bpf_mprog_read (entry, i, &fp, &cp); +bpf_mprog_copy (fpp, &cpp[j], fp, cp); + j++; + } +bpf_mprog_commit_cp (peer, cpp); +bpf_mprog_dec (peer); +bpf_mprog_mark_ref (peer, dtuple); +return bpf_mprog_total (peer) ? +BPF_MPROG_SWAP :BPF_MPROG_FREE ;
for delete it's also a bit unclear to me. We are deleting some specific spot, so idx should be a valid [0, N) value, no? Then why the bpf_mprog_tuple_confirm() has this special <= first and idx >= last handling?
Deletion should be similar to instertion, just the shift is in the other direction. And then setting NULLs at N-1 position to ensure proper NULL termination of fp array.
Quoted 26 lines
kernel/bpf/mprog.c new file +26
-0+} + +/* In bpf_mprog_pos_*() we evaluate the target position for the BPF +* program/link that needs to be replaced, inserted or deleted for +* each "rule" independently. If all rules agree on that position +* or existing element, then enact replacement, addition or deletion. +* If this is not the case, then the request cannot be satisfied and +* we bail out with an error. +*/ +static int bpf_mprog_pos_exact (struct bpf_mprog_entry *entry, +struct bpf_tuple *tuple) +{ +struct bpf_mprog_fp *fp; +struct bpf_mprog_cp *cp; +int i; + +for (i =0 ; i <bpf_mprog_total (entry); i++) { +bpf_mprog_read (entry, i, &fp, &cp); +if (tuple->prog ==READ_ONCE (fp->prog)) +return tuple->link == cp->link ? i : -EBUSY ; + } +return -ENOENT ; +} + +static int bpf_mprog_pos_before (struct bpf_mprog_entry *entry, +struct bpf_tuple *tuple)
kernel/bpf/mprog.c line 224
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+{ +struct bpf_mprog_fp *fp; +struct bpf_mprog_cp *cp; +int i; + +for (i =0 ; i <bpf_mprog_total (entry); i++) { +bpf_mprog_read (entry, i, &fp, &cp); +if (tuple->prog ==READ_ONCE (fp->prog) && + (!tuple->link || tuple->link == cp->link)) +return i -1 ;
taking all the above into account, this should just return i;
Quoted 9 lines
kernel/bpf/mprog.c new file +9
-0+ } +return tuple->prog ? -ENOENT :-1 ; +} + +static int bpf_mprog_pos_after (struct bpf_mprog_entry *entry, +struct bpf_tuple *tuple) +{ +struct bpf_mprog_fp *fp; +struct bpf_mprog_cp *cp;
kernel/bpf/mprog.c line 243
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+int i; + +for (i =0 ; i <bpf_mprog_total (entry); i++) { +bpf_mprog_read (entry, i, &fp, &cp); +if (tuple->prog ==READ_ONCE (fp->prog) && + (!tuple->link || tuple->link == cp->link)) +return i +1 ; + } +return tuple->prog ? -ENOENT :bpf_mprog_total (entry); +}
I actually wonder if it would be simpler to not have _exact, _before, and _after variant. Instead do generic find of a tuple. And then outside of that, depending on BPF_F_BEFORE/BPF_F_AFTER/BPF_F_REPLACE just adjust returned position (if item is found) to either keep it as is for BPF_F_BEFORE and BPF_F_REPLACE, or adjust it +1 for BPF_F_AFTER
kernel/bpf/mprog.c line 256
Quoted 13 lines
kernel/bpf/mprog.c new file +13
-0+ +int bpf_mprog_attach (struct bpf_mprog_entry *entry,struct bpf_prog *prog_new, +struct bpf_link *link,struct bpf_prog *prog_old, +u32 flags,u32 object,u64 revision) +{ +struct bpf_tuple rtuple, ntuple = { + .prog = prog_new, + .link = link, + }, otuple = { + .prog = prog_old, + .link = link, + }; +int ret, idx =-2 , tidx;
so here I'd init idx to some "impossible" error, like -ERANGE (to pair with -EDOM ;)
Quoted 23 lines
kernel/bpf/mprog.c new file +23
-0+ +if (revision && revision !=bpf_mprog_revision (entry)) +return -ESTALE ; +if (bpf_mprog_exists (entry, prog_new)) +return -EEXIST ; + ret =bpf_mprog_tuple_relative (&rtuple, object, + flags & ~BPF_F_REPLACE , + prog_new->type); +if (ret) +return ret; +if (flags &BPF_F_REPLACE ) { + tidx =bpf_mprog_pos_exact (entry, &otuple); +if (tidx <0 ) { + ret = tidx; + goto out; + } + idx = tidx; + } +if (flags &BPF_F_BEFORE ) { + tidx =bpf_mprog_pos_before (entry, &rtuple); +if (tidx <-1 || (idx >=-1 && tidx != idx)) { + ret = tidx <-1 ? tidx : -EDOM ; + goto out;
kernel/bpf/mprog.c line 289
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+ } + idx = tidx; + } +if (flags &BPF_F_AFTER ) { + tidx =bpf_mprog_pos_after (entry, &rtuple); +if (tidx <-1 || (idx >=-1 && tidx != idx)) { + ret = tidx <0 ? tidx : -EDOM ; + goto out; + } + idx = tidx;
and then here just have special casing for -ERANGE, and otherwise treat anything else negative as error
tidx = bpf_mprog_pos_exact(entry, &rtuple);
/* and adjust +1 for BPF_F_AFTER */
if (tidx >= 0)
tidx += 1;
if (idx != -ERANGE && tidx != idx) {
ret = tidx < 0 ? tidx : -EDOM;
goto out;
}idx = tidx;
Quoted 22 lines
kernel/bpf/mprog.c new file +22
-0+ } +if (idx <-1 ) { +if (rtuple.prog || flags) { + ret = -EINVAL ; + goto out; + } + idx =bpf_mprog_total (entry); + flags =BPF_F_AFTER ; + } +if (idx >=bpf_mprog_max ()) { + ret = -EDOM ; + goto out; + } +if (flags &BPF_F_REPLACE ) + ret =bpf_mprog_replace (entry, &ntuple, idx); +else + ret =bpf_mprog_insert (entry, &ntuple, idx, flags); +out: +bpf_mprog_tuple_put (&rtuple); +return ret; +} +
[...]
Daniel Borkmann
maintainerRe: [PATCH bpf-next v4 1/8] bpf: Add generic attach/detach/query API for multi-progs
Quoted 1 line · Andrii Nakryiko
On Mon, Jul 10, 2023 at 1:12 PM Daniel Borkmann <[email protected]> wrote:
[...]
Quoted 7 lines
include/linux/bpf_mprog.h line 214
Quoted 4 lines
include/linux/bpf_mprog.h new file +4
-0+static inline int bpf_mprog_max (void ) +{ +return ARRAY_SIZE (((struct bpf_mprog_entry *)NULL )->fp_items) -1 ; +}so we can only add
BPF_MPROG_MAX- 1 progs, right? I presume the last entry is presumed to be always NULL, right?
Correct.
Quoted 35 lines
Quoted 8 lines
include/linux/bpf_mprog.h new file +8
-0+static inline int bpf_mprog_total (struct bpf_mprog_entry *entry) +{ +int total = entry->parent->count; + +WARN_ON_ONCE (total >bpf_mprog_max ()); +return total; +} +[...]
Quoted 24 lines
kernel/bpf/Makefile +1
-1@@ -12,7 +12,7 @@ obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o -obj-$(CONFIG_BPF_SYSCALL) += disasm.o +obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o obj-$(CONFIG_BPF_JIT) += trampoline.o obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o obj-$(CONFIG_BPF_JIT) += dispatcher.okernel/bpf/mprog.c new file +5
-0@@ -0,0 +1,427 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Isovalent */ + +#include <linux/bpf.h> +#include <linux/bpf_mprog.h>
Quoted 8 lines
kernel/bpf/mprog.c line 8
Quoted 3 lines
kernel/bpf/mprog.c new file +3
-0+ +static int bpf_mprog_link(struct bpf_tuple *tuple, + u32 object, u32 flags,so I tried to get used to this "object" notation, but I think it's still awkwards and keeps me asking "what is this really" every single time I read this. I wonder if something like "
fd_or_id" as a name would make it more obvious?
Ok, fixed it in the v5.
[...]
Quoted 65 lines
Quoted 19 lines
kernel/bpf/mprog.c new file +19
-0+struct bpf_mprog_fp *fp, *fpp; +struct bpf_mprog_entry *peer; + + peer =bpf_mprog_peer (entry); +bpf_mprog_entry_clear (peer); +if (idx <0 ) { +bpf_mprog_read_fp (peer, j, &fpp); +bpf_mprog_write_fp (fpp, ntuple); +bpf_mprog_write_cp (&cpp[j], ntuple); + j++; + } +for (i =0 ; i <= total; i++) { +bpf_mprog_read_fp (peer, j, &fpp); +if (idx == i && (flags &BPF_F_AFTER )) { +bpf_mprog_write (fpp, &cpp[j], ntuple); + j++; +bpf_mprog_read_fp (peer, j, &fpp); + } +if (i < total) {kernel/bpf/mprog.c line 124
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+bpf_mprog_read (entry, i, &fp, &cp); +bpf_mprog_copy (fpp, &cpp[j], fp, cp); + j++; + } +if (idx == i && (flags &BPF_F_BEFORE )) { +bpf_mprog_read_fp (peer, j, &fpp); +bpf_mprog_write (fpp, &cpp[j], ntuple); + j++; + } + }sorry if I miss some subtle point, but I wonder why this is so complicated? I think this choice of idx == -1 meaning prepend is leading to this complication. It's not also clear why there is this
BPF_F_AFTERvsBPF_F_BEFOREdistinction when we already determined a position where new program has to be inserted (so after or before should be irrelevant).Please let me know why the below doesn't work.
Let's define that idx is the position where new prog/link tuple has to be inserted. It can be in the range [0, N], where N is number of programs currently in the
mprog_peer. Note that N is inclusive above.The algorithm for insertion is simple: everything currently at entry->fp_items[idx] and after gets shifted. And we can do it with a simple memmove:
memmove(peer->fp_items + idx + 1,
peer->fp_iters+ idx, (bpf_mprog_total(entry)- idx) *sizeof(struct bpf_mprof_fp)); /* similar memmove for cp_items/cpp array, of course */ /* now set new prog at peer->fp_items[idx] */The above should replace entire above for loop and that extra if before the loop. And it should work for corner cases:
- idx == 0 (prepend), will shift everything to the right, and put
new prog at position 0. Exactly what we wanted.
- idx == N (append), will shift nothing (that memmov should be a
no-op because size is zero, total == idx == N)We just need to make sure that the above shift won't overwrite the very last NULL. So
bpf_mprog_total()should be <BPF_MPROG_MAX- 2 before all this.Seems as simple as that, is there any complication I skimmed over?
[...]
Quoted 36 lines
Quoted 20 lines
kernel/bpf/mprog.c new file +20
-0+static int bpf_mprog_delete (struct bpf_mprog_entry *entry, +struct bpf_tuple *dtuple,int idx) +{ +int i =0 , j, ret, total =bpf_mprog_total (entry); +struct bpf_mprog_cp *cp, cpp[BPF_MPROG_MAX ] = {}; +struct bpf_mprog_fp *fp, *fpp; +struct bpf_mprog_entry *peer; + + ret =bpf_mprog_tuple_confirm (entry, dtuple, idx); +if (ret) +return ret; + peer =bpf_mprog_peer (entry); +bpf_mprog_entry_clear (peer); +if (idx <0 ) + i++; +if (idx == total) + total--; +for (j =0 ; i < total; i++) { +if (idx == i) +continue ;kernel/bpf/mprog.c line 188
Quoted 10 lines
kernel/bpf/mprog.c new file +10
-0+bpf_mprog_read_fp (peer, j, &fpp); +bpf_mprog_read (entry, i, &fp, &cp); +bpf_mprog_copy (fpp, &cpp[j], fp, cp); + j++; + } +bpf_mprog_commit_cp (peer, cpp); +bpf_mprog_dec (peer); +bpf_mprog_mark_ref (peer, dtuple); +return bpf_mprog_total (peer) ? +BPF_MPROG_SWAP :BPF_MPROG_FREE ;for delete it's also a bit unclear to me. We are deleting some specific spot, so idx should be a valid [0, N) value, no? Then why the
bpf_mprog_tuple_confirm()has this special <= first and idx >= last handling?
Quoted 3 lines
Deletion should be similar to instertion, just the shift is in the other direction. And then setting NULLs at N-1 position to ensure proper NULL termination of fp array.
Agree, the naming was suboptimal and I adapted this slightly in v5. It's picking the elements when no deletion fd was selected, but rather delete from front/back or relative to some element, so it needs to fetch the prog.
[...]
Quoted 7 lines
and then here just have special casing for
-ERANGE, and otherwise treat anything else negative as errortidx =
bpf_mprog_pos_exact(entry, &rtuple);
/* and adjust +1 forBPF_F_AFTER*/
if (tidx >= 0)
Quoted 6 lines
tidx += 1; if (idx != -ERANGE && tidx != idx) { ret = tidx < 0 ? tidx : -EDOM; goto out; }idx = tidx;
This looks much less intuitive to me given replace and delete case need exact position, just not the relative insertion. I reworked this also with the memmove in v5, but kept the more obvious _exact/before/after ones.
Thanks a lot for the feedback!
Quoted 25 lines
Quoted 22 lines
kernel/bpf/mprog.c new file +22
-0+ } +if (idx <-1 ) { +if (rtuple.prog || flags) { + ret = -EINVAL ; + goto out; + } + idx =bpf_mprog_total (entry); + flags =BPF_F_AFTER ; + } +if (idx >=bpf_mprog_max ()) { + ret = -EDOM ; + goto out; + } +if (flags &BPF_F_REPLACE ) + ret =bpf_mprog_replace (entry, &ntuple, idx); +else + ret =bpf_mprog_insert (entry, &ntuple, idx, flags); +out: +bpf_mprog_tuple_put (&rtuple); +return ret; +} +[...]
Daniel Borkmann
maintainerPATCH v4 2/8 bpf: Add fd-based tcx multi-prog infra with link support
In reply to Daniel BorkmannThis work refactors and adds a lightweight extension ("tcx") to the tc BPF ingress and egress data path side for allowing BPF program management based on fds via bpf() syscall through the newly added generic multi-prog API. The main goal behind this work which we also presented at LPC [0] last year and a recent update at LSF/MM/BPF this year [3] is to support long-awaited BPF link functionality for tc BPF programs, which allows for a model of safe ownership and program detachment.
Given the rise in tc BPF users in cloud native environments, this becomes necessary to avoid hard to debug incidents either through stale leftover programs or 3rd party applications accidentally stepping on each others toes. As a recap, a BPF link represents the attachment of a BPF program to a BPF hook point. The BPF link holds a single reference to keep BPF program alive. Moreover, hook points do not reference a BPF link, only the application's fd or pinning does. A BPF link holds meta-data specific to attachment and implements operations for link creation, (atomic) BPF program update, detachment and introspection. The motivation for BPF links for tc BPF programs is multi-fold, for example:
- From Meta: "It's especially important for applications that are deployed fleet-wide and that don't "control" hosts they are deployed to. If such application crashes and no one notices and does anything about that, BPF program will keep running draining resources or even just, say, dropping packets. We at FB had outages due to such permanent BPF attachment semantics. With fd-based BPF link we are getting a framework, which allows safe, auto-detachable behavior by default, unless application explicitly opts in by pinning the BPF link." [1]
- From Cilium-side the tc BPF programs we attach to host-facing veth devices and phys devices build the core datapath for Kubernetes Pods, and they implement forwarding, load-balancing, policy, EDT-management, etc, within BPF. Currently there is no concept of 'safe' ownership, e.g. we've recently experienced hard-to-debug issues in a user's staging environment where another Kubernetes application using tc BPF attached to the same prio/handle of cls_bpf, accidentally wiping all Cilium-based BPF programs from underneath it. The goal is to establish a clear/safe ownership model via links which cannot accidentally be overridden. [0,2]
BPF links for tc can co-exist with non-link attachments, and the semantics are in line also with XDP links: BPF links cannot replace other BPF links, BPF links cannot replace non-BPF links, non-BPF links cannot replace BPF links and lastly only non-BPF links can replace non-BPF links. In case of Cilium, this would solve mentioned issue of safe ownership model as 3rd party applications would not be able to accidentally wipe Cilium programs, even if they are not BPF link aware.
Earlier attempts [4] have tried to integrate BPF links into core tc machinery to solve cls_bpf, which has been intrusive to the generic tc kernel API with extensions only specific to cls_bpf and suboptimal/complex since cls_bpf could be wiped from the qdisc also. Locking a tc BPF program in place this way, is getting into layering hacks given the two object models are vastly different.
We instead implemented the tcx (tc 'express') layer which is an fd-based tc BPF attach API, so that the BPF link implementation blends in naturally similar to other link types which are fd-based and without the need for changing core tc internal APIs. BPF programs for tc can then be successively migrated from classic cls_bpf to the new tc BPF link without needing to change the program's source code, just the BPF loader mechanics for attaching is sufficient.
For the current tc framework, there is no change in behavior with this change and neither does this change touch on tc core kernel APIs. The gist of this patch is that the ingress and egress hook have a lightweight, qdisc-less extension for BPF to attach its tc BPF programs, in other words, a minimal entry point for tc BPF. The name tcx has been suggested from discussion of earlier revisions of this work as a good fit, and to more easily differ between the classic cls_bpf attachment and the fd-based one.
For the ingress and egress tcx points, the device holds a cache-friendly array with program pointers which is separated from control plane (slow-path) data. Earlier versions of this work used priority to determine ordering and expression of dependencies similar as with classic tc, but it was challenged that for something more future-proof a better user experience is required. Hence this resulted in the design and development of the generic attach/detach/query API for multi-progs. See prior patch with its discussion on the API design. tcx is the first user and later we plan to integrate also others, for example, one candidate is multi-prog support for XDP which would benefit and have the same 'look and feel' from API perspective.
The goal with tcx is to have maximum compatibility to existing tc BPF programs, so they don't need to be rewritten specifically. Compatibility to call into classic tcf_classify() is also provided in order to allow successive migration or both to cleanly co-exist where needed given its all one logical tc layer and the tcx plus classic tc cls/act build one logical overall processing pipeline.
tcx supports the simplified return codes TCX_NEXT which is non-terminating (go to next program) and terminating ones with TCX_PASS, TCX_DROP, TCX_REDIRECT. The fd-based API is behind a static key, so that when unused the code is also not entered. The struct tcx_entry's program array is currently static, but could be made dynamic if necessary at a point in future. The a/b pair swap design has been chosen so that for detachment there are no allocations which otherwise could fail.
The work has been tested with tc-testing selftest suite which all passes, as well as the tc BPF tests from the BPF CI, and also with Cilium's L4LB.
Kudos also to Nikolay Aleksandrov and Martin Lau for in-depth early reviews of this work.
[0] https://lpc.events/event/16/contributions/1353/
[1] https://lore.kernel.org/bpf/CAEf4BzbokCJN33Nw_kg82sO=xppXnKWEncGTWCTB9vGCmLB6pw@mail.gmail.com
[2] https://colocatedeventseu2023.sched.com/event/1Jo6O/tales-from-an-ebpf-programs-murder-mystery-hemanth-malla-guillaume-fournier-datadog
[3] http://vger.kernel.org/bpfconf2023_material/tcx_meta_netdev_borkmann.pdf
[4] https://lore.kernel.org/bpf/[email protected]
--- MAINTAINERS | 4 +- include/linux/bpf_mprog.h | 9 + include/linux/netdevice.h | 15 +- include/linux/skbuff.h | 4 +- include/net/sch_generic.h | 2 +- include/net/tcx.h | 199 +++++++++++++++++++ include/uapi/linux/bpf.h | 34 +++- kernel/bpf/Kconfig | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/syscall.c | 83 ++++++-- kernel/bpf/tcx.c | 351 +++++++++++++++++++++++++++++++++ net/Kconfig | 5 + net/core/dev.c | 267 +++++++++++++++---------- net/core/filter.c | 4 +- net/sched/Kconfig | 4 +- net/sched/sch_ingress.c | 61 +++++- tools/include/uapi/linux/bpf.h | 34 +++- 17 files changed, 934 insertions(+), 144 deletions(-) create mode 100644 include/net/tcx.h create mode 100644 kernel/bpf/tcx.c
MAINTAINERS +3 -1
@@ -3830,13 +3830,15 @@ L: [email protected] S: Maintained F: kernel/bpf/bpf_struct* -BPF [NETWORKING] (tc BPF, sock_addr) +BPF [NETWORKING] (tcx & tc BPF, sock_addr) M: Martin KaFai Lau <[email protected]> M: Daniel Borkmann <[email protected]> R: John Fastabend <[email protected]> L: [email protected] L: [email protected] S: Maintained +F: include/net/tcx.h +F: kernel/bpf/tcx.c F: net/core/filter.c F: net/sched/act_bpf.c F: net/sched/cls_bpf.c
include/linux/bpf_mprog.h +9 -0
@@ -340,4 +340,13 @@ int bpf_mprog_detach(struct bpf_mprog_entry *entry, struct bpf_prog *prog,int bpf_mprog_query (const union bpf_attr *attr,union bpf_attr __user *uattr,struct bpf_mprog_entry *entry); +static inline bool bpf_mprog_supported (enum bpf_prog_type type) +{ +switch (type) { +case BPF_PROG_TYPE_SCHED_CLS : +return true; +default : +return false; + } +}#endif /* __BPF_MPROG_H */
include/linux/netdevice.h +6 -9
@@ -1930,8 +1930,7 @@ enum netdev_ml_priv_type {* * @rx_handler: handler for received packets * @rx_handler_data: XXX: need comments on this one -* @miniq_ingress: ingress/clsact qdisc specific data for -* ingress processing +* @tcx_ingress: BPF & clsact qdisc specific data for ingress processing * @ingress_queue: XXX: need comments on this one * @nf_hooks_ingress: netfilter hooks executed for ingress packets * @broadcast: hw bcast address
@@ -1952,8 +1951,7 @@ enum netdev_ml_priv_type {* @xps_maps: all CPUs/RXQs maps for XPS device * * @xps_maps: XXX: need comments on this one -* @miniq_egress: clsact qdisc specific data for -* egress processing +* @tcx_egress: BPF & clsact qdisc specific data for egress processing * @nf_hooks_egress: netfilter hooks executed for egress packets * @qdisc_hash: qdisc hash table * @watchdog_timeo: Represents the timeout that is used by
@@ -2252,9 +2250,8 @@ struct net_device {unsigned int gro_ipv4_max_size;rx_handler_func_t __rcu *rx_handler;void __rcu *rx_handler_data; - -#ifdef CONFIG_NET_CLS_ACT -struct mini_Qdisc __rcu *miniq_ingress; +#ifdef CONFIG_NET_XGRESS +struct bpf_mprog_entry __rcu *tcx_ingress;#endif struct netdev_queue __rcu *ingress_queue;#ifdef CONFIG_NETFILTER_INGRESS
@@ -2282,8 +2279,8 @@ struct net_device {#ifdef CONFIG_XPS struct xps_dev_maps __rcu *xps_maps[XPS_MAPS_MAX ];#endif -#ifdef CONFIG_NET_CLS_ACT -struct mini_Qdisc __rcu *miniq_egress; +#ifdef CONFIG_NET_XGRESS +struct bpf_mprog_entry __rcu *tcx_egress;#endif #ifdef CONFIG_NETFILTER_EGRESS struct nf_hook_entries __rcu *nf_hooks_egress;
include/linux/skbuff.h +2 -2
@@ -944,7 +944,7 @@ struct sk_buff {__u8 __mono_tc_offset[0 ];/* public: */ __u8 mono_delivery_time:1; /* See SKB_MONO_DELIVERY_TIME_MASK */ -#ifdef CONFIG_NET_CLS_ACT +#ifdef CONFIG_NET_XGRESS __u8 tc_at_ingress:1; /* See TC_AT_INGRESS_MASK */ __u8 tc_skip_classify:1;#endif
@@ -993,7 +993,7 @@ struct sk_buff { __u8 csum_not_inet:1;#endif -#ifdef CONFIG_NET_SCHED +#if defined(CONFIG_NET_SCHED ) || defined(CONFIG_NET_XGRESS )__u16 tc_index;/* traffic control index */ #endif
include/net/sch_generic.h +1 -1
@@ -703,7 +703,7 @@ int skb_do_redirect(struct sk_buff *);static inline bool skb_at_tc_ingress (const struct sk_buff *skb) { -#ifdef CONFIG_NET_CLS_ACT +#ifdef CONFIG_NET_XGRESS return skb->tc_at_ingress;#else return false;
include/net/tcx.h new file +199 -0
199 lines not on this page · raw message
include/uapi/linux/bpf.h +30 -4
@@ -1036,6 +1036,8 @@ enum bpf_attach_type {BPF_LSM_CGROUP ,BPF_STRUCT_OPS ,BPF_NETFILTER , +BPF_TCX_INGRESS , +BPF_TCX_EGRESS , __MAX_BPF_ATTACH_TYPE };
@@ -1053,7 +1055,7 @@ enum bpf_link_type {BPF_LINK_TYPE_KPROBE_MULTI =8 ,BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 , - +BPF_LINK_TYPE_TCX =11 , MAX_BPF_LINK_TYPE, };
@@ -1559,13 +1561,13 @@ union bpf_attr {__u32 map_fd;/* struct_ops to attach */ };union { -__u32 target_fd;/* object to attach to */ -__u32 target_ifindex;/* target ifindex */ +__u32 target_fd;/* target object to attach to or ... */ +__u32 target_ifindex;/* target ifindex */ };__u32 attach_type;/* attach type */ __u32 flags;/* extra flags */ union { -__u32 target_btf_id;/* btf_id of target to attach to */ +__u32 target_btf_id;/* btf_id of target to attach to */ struct {__aligned_u64 iter_info;/* extra bpf_iter_link_info */ __u32 iter_info_len;/* iter_info length */
@@ -1599,6 +1601,13 @@ union bpf_attr {__s32 priority;__u32 flags; } netfilter; +struct { +union { +__u32 relative_fd; +__u32 relative_id; + }; +__u64 expected_revision; + } tcx; }; } link_create;
@@ -6207,6 +6216,19 @@ struct bpf_sock_tuple { }; }; +/* (Simplified) user return codes for tcx prog type. +* A valid tcx program must return one of these defined values. All other +* return codes are reserved for future use. Must remain compatible with +* their TC_ACT_* counter-parts. For compatibility in behavior, unknown +* return codes are mapped to TCX_NEXT. +*/ +enum tcx_action_base { +TCX_NEXT =-1 , +TCX_PASS =0 , +TCX_DROP =2 , +TCX_REDIRECT =7 , +}; +struct bpf_xdp_sock {__u32 queue_id; };
@@ -6459,6 +6481,10 @@ struct bpf_link_info {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 ifindex; +__u32 attach_type; + } tcx; }; } __attribute__((aligned(8)));
kernel/bpf/Kconfig +1 -0
@@ -31,6 +31,7 @@ config BPF_SYSCALL select TASKS_TRACE_RCU select BINARY_PRINTF select NET_SOCK_MSG if NET + select NET_XGRESS if NET select PAGE_POOL if NET default n help
kernel/bpf/Makefile +1 -0
@@ -21,6 +21,7 @@ obj-$(CONFIG_BPF_SYSCALL) += devmap.o obj-$(CONFIG_BPF_SYSCALL) += cpumap.o obj-$(CONFIG_BPF_SYSCALL) += offload.o obj-$(CONFIG_BPF_SYSCALL) += net_namespace.o +obj-$(CONFIG_BPF_SYSCALL) += tcx.o endif ifeq ($(CONFIG_PERF_EVENTS),y) obj-$(CONFIG_BPF_SYSCALL) += stackmap.o
kernel/bpf/syscall.c +70 -13
169 lines not on this page · raw message
kernel/bpf/tcx.c new file +351 -0
351 lines not on this page · raw message
net/Kconfig +5 -0
@@ -52,6 +52,11 @@ config NET_INGRESS config NET_EGRESS bool +config NET_XGRESS + select NET_INGRESS + select NET_EGRESS + bool + config NET_REDIRECT bool
net/core/dev.c +165 -102
335 lines not on this page · raw message
net/core/filter.c +2 -2
@@ -9312,7 +9312,7 @@ static struct bpf_insn *bpf_convert_tstamp_read(const struct bpf_prog *prog,__u8 value_reg = si->dst_reg;__u8 skb_reg = si->src_reg; -#ifdef CONFIG_NET_CLS_ACT +#ifdef CONFIG_NET_XGRESS /* If the tstamp_type is read, * the bpf prog is aware the tstamp could have delivery time. * Thus, read skb->tstamp as is if tstamp_type_access is true.
@@ -9346,7 +9346,7 @@ static struct bpf_insn *bpf_convert_tstamp_write(const struct bpf_prog *prog,__u8 value_reg = si->src_reg;__u8 skb_reg = si->dst_reg; -#ifdef CONFIG_NET_CLS_ACT +#ifdef CONFIG_NET_XGRESS /* If the tstamp_type is read, * the bpf prog is aware the tstamp could have delivery time. * Thus, write skb->tstamp as is if tstamp_type_access is true.
net/sched/Kconfig +2 -2
@@ -347,8 +347,7 @@ config NET_SCH_FQ_PIE config NET_SCH_INGRESS tristate "Ingress/classifier-action Qdisc" depends on NET_CLS_ACT - select NET_INGRESS - select NET_EGRESS + select NET_XGRESS help Say Y here if you want to use classifiers for incoming and/or outgoing packets. This qdisc doesn't do anything else besides running classifiers,
@@ -679,6 +678,7 @@ config NET_EMATCH_IPT config NET_CLS_ACT bool "Actions" select NET_CLS + select NET_XGRESS help Say Y here if you want to use traffic control actions. Actions get attached to classifiers and are invoked after a successful
net/sched/sch_ingress.c +57 -4
119 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +30 -4
@@ -1036,6 +1036,8 @@ enum bpf_attach_type {BPF_LSM_CGROUP ,BPF_STRUCT_OPS ,BPF_NETFILTER , +BPF_TCX_INGRESS , +BPF_TCX_EGRESS , __MAX_BPF_ATTACH_TYPE };
@@ -1053,7 +1055,7 @@ enum bpf_link_type {BPF_LINK_TYPE_KPROBE_MULTI =8 ,BPF_LINK_TYPE_STRUCT_OPS =9 ,BPF_LINK_TYPE_NETFILTER =10 , - +BPF_LINK_TYPE_TCX =11 , MAX_BPF_LINK_TYPE, };
@@ -1559,13 +1561,13 @@ union bpf_attr {__u32 map_fd;/* struct_ops to attach */ };union { -__u32 target_fd;/* object to attach to */ -__u32 target_ifindex;/* target ifindex */ +__u32 target_fd;/* target object to attach to or ... */ +__u32 target_ifindex;/* target ifindex */ };__u32 attach_type;/* attach type */ __u32 flags;/* extra flags */ union { -__u32 target_btf_id;/* btf_id of target to attach to */ +__u32 target_btf_id;/* btf_id of target to attach to */ struct {__aligned_u64 iter_info;/* extra bpf_iter_link_info */ __u32 iter_info_len;/* iter_info length */
@@ -1599,6 +1601,13 @@ union bpf_attr {__s32 priority;__u32 flags; } netfilter; +struct { +union { +__u32 relative_fd; +__u32 relative_id; + }; +__u64 expected_revision; + } tcx; }; } link_create;
@@ -6207,6 +6216,19 @@ struct bpf_sock_tuple { }; }; +/* (Simplified) user return codes for tcx prog type. +* A valid tcx program must return one of these defined values. All other +* return codes are reserved for future use. Must remain compatible with +* their TC_ACT_* counter-parts. For compatibility in behavior, unknown +* return codes are mapped to TCX_NEXT. +*/ +enum tcx_action_base { +TCX_NEXT =-1 , +TCX_PASS =0 , +TCX_DROP =2 , +TCX_REDIRECT =7 , +}; +struct bpf_xdp_sock {__u32 queue_id; };
@@ -6459,6 +6481,10 @@ struct bpf_link_info {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 ifindex; +__u32 attach_type; + } tcx; }; } __attribute__((aligned(8)));
Signature · 1 line
2.34.1
Daniel Borkmann
maintainerPATCH v4 3/8 libbpf: Add opts-based attach/detach/query API for tcx
In reply to Daniel BorkmannExtend libbpf attach opts and add a new detach opts API so this can be used to add/remove fd-based tcx BPF programs. The old-style bpf_prog_detach() and bpf_prog_detach2() APIs are refactored to reuse the new bpf_prog_detach_opts() internally.
The bpf_prog_query_opts() API got extended to be able to handle the new link_ids, link_attach_flags and revision fields.
For concrete usage examples, see the extensive selftests that have been developed as part of this series.
--- tools/lib/bpf/bpf.c | 105 +++++++++++++++++++++++++-------------- tools/lib/bpf/bpf.h | 92 ++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.c | 12 +++-- tools/lib/bpf/libbpf.map | 1 + 4 files changed, 157 insertions(+), 53 deletions(-)
tools/lib/bpf/bpf.c +69 -36
@@ -629,55 +629,87 @@ int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type,return bpf_prog_attach_opts (prog_fd, target_fd, type, &opts); } -int bpf_prog_attach_opts (int prog_fd,int target_fd, -enum bpf_attach_type type, -const struct bpf_prog_attach_opts *opts) +int bpf_prog_attach_opts (int prog_fd,int target, +enum bpf_attach_type type, +const struct bpf_prog_attach_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); + const size_t attr_sz = offsetofend(union bpf_attr, expected_revision); +__u32 relative_id, flags;union bpf_attr attr; -int ret; +int ret, relative;if (!OPTS_VALID (opts, bpf_prog_attach_opts))return libbpf_err (-EINVAL ); + relative_id =OPTS_GET (opts, relative_id,0 ); + relative =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + flags |=BPF_F_ID ; + } +memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_bpf_fd = prog_fd; - attr.attach_type = type; - attr.attach_flags =OPTS_GET (opts, flags,0 ); - attr.replace_bpf_fd =OPTS_GET (opts, replace_prog_fd,0 ); + attr.target_fd = target; + attr.attach_bpf_fd = prog_fd; + attr.attach_type = type; + attr.attach_flags = flags; + attr.relative_fd = relative; + attr.replace_bpf_fd =OPTS_GET (opts, replace_fd,0 ); + attr.expected_revision =OPTS_GET (opts, expected_revision,0 ); ret =sys_bpf (BPF_PROG_ATTACH , &attr, attr_sz);return libbpf_err_errno (ret); } -int bpf_prog_detach (int target_fd,enum bpf_attach_type type) +int bpf_prog_detach_opts (int prog_fd,int target, +enum bpf_attach_type type, +const struct bpf_prog_detach_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); + const size_t attr_sz = offsetofend(union bpf_attr, expected_revision); +__u32 relative_id, flags;union bpf_attr attr; -int ret; +int ret, relative; + +if (!OPTS_VALID (opts, bpf_prog_detach_opts)) +return libbpf_err (-EINVAL ); + + relative_id =OPTS_GET (opts, relative_id,0 ); + relative =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + flags |=BPF_F_ID ; + }memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_type = type; + attr.target_fd = target; + attr.attach_bpf_fd = prog_fd; + attr.attach_type = type; + attr.attach_flags = flags; + attr.relative_fd = relative; + attr.expected_revision =OPTS_GET (opts, expected_revision,0 ); ret =sys_bpf (BPF_PROG_DETACH , &attr, attr_sz);return libbpf_err_errno (ret); } -int bpf_prog_detach2 (int prog_fd,int target_fd,enum bpf_attach_type type) +int bpf_prog_detach (int target_fd,enum bpf_attach_type type) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); -union bpf_attr attr; -int ret; - -memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_bpf_fd = prog_fd; - attr.attach_type = type; +return bpf_prog_detach_opts (0 , target_fd, type,NULL ); +} - ret =sys_bpf (BPF_PROG_DETACH , &attr, attr_sz); -return libbpf_err_errno (ret); +int bpf_prog_detach2 (int prog_fd,int target_fd,enum bpf_attach_type type) +{ +return bpf_prog_detach_opts (prog_fd, target_fd, type,NULL ); } int bpf_link_create(int prog_fd, int target_fd,
@@ -841,8 +873,7 @@ int bpf_iter_create(int link_fd)return libbpf_err_errno (fd); } -int bpf_prog_query_opts (int target_fd, -enum bpf_attach_type type, +int bpf_prog_query_opts (int target,enum bpf_attach_type type,struct bpf_prog_query_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, query);
@@ -853,18 +884,20 @@ int bpf_prog_query_opts(int target_fd,return libbpf_err (-EINVAL );memset (&attr,0 , attr_sz); - - attr.query.target_fd = target_fd; - attr.query.attach_type = type; - attr.query.query_flags =OPTS_GET (opts, query_flags,0 ); - attr.query.prog_cnt =OPTS_GET (opts, prog_cnt,0 ); - attr.query.prog_ids =ptr_to_u64 (OPTS_GET (opts, prog_ids,NULL )); - attr.query.prog_attach_flags =ptr_to_u64 (OPTS_GET (opts, prog_attach_flags,NULL )); + attr.query.target_fd = target; + attr.query.attach_type = type; + attr.query.query_flags =OPTS_GET (opts, query_flags,0 ); + attr.query.count =OPTS_GET (opts, count,0 ); + attr.query.prog_ids =ptr_to_u64 (OPTS_GET (opts, prog_ids,NULL )); + attr.query.link_ids =ptr_to_u64 (OPTS_GET (opts, link_ids,NULL )); + attr.query.prog_attach_flags =ptr_to_u64 (OPTS_GET (opts, prog_attach_flags,NULL )); + attr.query.link_attach_flags =ptr_to_u64 (OPTS_GET (opts, link_attach_flags,NULL )); ret =sys_bpf (BPF_PROG_QUERY , &attr, attr_sz);OPTS_SET (opts, attach_flags, attr.query.attach_flags); -OPTS_SET (opts, prog_cnt, attr.query.prog_cnt); +OPTS_SET (opts, revision, attr.query.revision); +OPTS_SET (opts, count, attr.query.count);return libbpf_err_errno (ret); }
tools/lib/bpf/bpf.h +78 -14
113 lines not on this page · raw message
tools/lib/bpf/libbpf.c +9 -3
@@ -118,6 +118,8 @@ static const char * const attach_type_name[] = { [BPF_TRACE_KPROBE_MULTI] = "trace_kprobe_multi", [BPF_STRUCT_OPS] = "struct_ops", [BPF_NETFILTER] = "netfilter", + [BPF_TCX_INGRESS] = "tcx_ingress", + [BPF_TCX_EGRESS] = "tcx_egress", };static const char *const link_type_name[] = {
@@ -8691,9 +8693,13 @@ static const struct bpf_sec_def section_defs[] = {SEC_DEF ("ksyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("kretsyscall+" ,KPROBE ,0 ,SEC_NONE , attach_ksyscall),SEC_DEF ("usdt+" ,KPROBE ,0 ,SEC_NONE , attach_usdt), -SEC_DEF ("tc" ,SCHED_CLS ,0 ,SEC_NONE ), -SEC_DEF ("classifier" ,SCHED_CLS ,0 ,SEC_NONE ), -SEC_DEF ("action" ,SCHED_ACT ,0 ,SEC_NONE ), +SEC_DEF ("tc/ingress" ,SCHED_CLS ,BPF_TCX_INGRESS ,SEC_NONE ),/* alias for tcx */ +SEC_DEF ("tc/egress" ,SCHED_CLS ,BPF_TCX_EGRESS ,SEC_NONE ),/* alias for tcx */ +SEC_DEF ("tcx/ingress" ,SCHED_CLS ,BPF_TCX_INGRESS ,SEC_NONE ), +SEC_DEF ("tcx/egress" ,SCHED_CLS ,BPF_TCX_EGRESS ,SEC_NONE ), +SEC_DEF ("tc" ,SCHED_CLS ,0 ,SEC_NONE ),/* deprecated / legacy, use tcx */ +SEC_DEF ("classifier" ,SCHED_CLS ,0 ,SEC_NONE ),/* deprecated / legacy, use tcx */ +SEC_DEF ("action" ,SCHED_ACT ,0 ,SEC_NONE ),/* deprecated / legacy, use tcx */ SEC_DEF ("tracepoint+" ,TRACEPOINT ,0 ,SEC_NONE , attach_tp),SEC_DEF ("tp+" ,TRACEPOINT ,0 ,SEC_NONE , attach_tp), SEC_DEF("raw_tracepoint+", RAW_TRACEPOINT, 0, SEC_NONE, attach_raw_tp),
tools/lib/bpf/libbpf.map +1 -0
@@ -396,4 +396,5 @@ LIBBPF_1.3.0 { global: bpf_obj_pin_opts; bpf_program__attach_netfilter; + bpf_prog_detach_opts; } LIBBPF_1.2.0;
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v4 3/8] libbpf: Add opts-based attach/detach/query API for tcx
Quoted 12 lines · Daniel Borkmann
Extend libbpf attach opts and add a new detach opts API so this can be used to add/remove fd-based tcx BPF programs. The old-style
bpf_prog_detach()andbpf_prog_detach2()APIs are refactored to reuse the newbpf_prog_detach_opts()internally.The
bpf_prog_query_opts()API got extended to be able to handle the newlink_ids,link_attach_flagsand revision fields.For concrete usage examples, see the extensive selftests that have been developed as part of this series.
Quoted 8 lines
Signed-off-by Daniel Borkmann <[email protected]>--- tools/lib/bpf/bpf.c | 105 +++++++++++++++++++++++++-------------- tools/lib/bpf/bpf.h | 92 ++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.c | 12 +++-- tools/lib/bpf/libbpf.map | 1 + 4 files changed, 157 insertions(+), 53 deletions(-)
Thanks for doc comments! Looks good, left a few nits with suggestions for simplifying code, but it's minor.
Quoted 22 lines
tools/lib/bpf/bpf.c +6
-5@@ -629,55 +629,87 @@ int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type,return bpf_prog_attach_opts (prog_fd, target_fd, type, &opts); } -int bpf_prog_attach_opts (int prog_fd,int target_fd, -enum bpf_attach_type type, -const struct bpf_prog_attach_opts *opts) +int bpf_prog_attach_opts (int prog_fd,int target, +enum bpf_attach_type type, +const struct bpf_prog_attach_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); + const size_t attr_sz = offsetofend(union bpf_attr, expected_revision); +__u32 relative_id, flags;union bpf_attr attr; -int ret; +int ret, relative;
tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 650
Quoted 10 lines
tools/lib/bpf/bpf.c +7
-0if (!OPTS_VALID (opts, bpf_prog_attach_opts))return libbpf_err (-EINVAL ); + relative_id =OPTS_GET (opts, relative_id,0 ); + relative =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (relative >0 && relative_id) +return libbpf_err (-EINVAL );
I left a comment in the next patch about this, I think it should be simple if (relative_fd && relative_id) { /* bad */ }. But see the next patch for why.
tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 654
Quoted 4 lines
tools/lib/bpf/bpf.c +4
-0+if (relative_id) { + relative = relative_id; + flags |=BPF_F_ID ; + }
it's a bit hard to follow as written (to me at least). How about a slight variation that has less in-place state update
int relative_fd, relative_id;
relative_fd = OPTS_GET(opts, relative_fd, 0);relative_id = OPTS_GET(opts, relative_id, 0);
/* only one of fd or id can be specified */
if (relative_fd && relative_id > 0)
return libbpf_err(-EINVAL);
... then see further below
tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 661
Quoted 12 lines
tools/lib/bpf/bpf.c +6
-5+memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_bpf_fd = prog_fd; - attr.attach_type = type; - attr.attach_flags =OPTS_GET (opts, flags,0 ); - attr.replace_bpf_fd =OPTS_GET (opts, replace_prog_fd,0 ); + attr.target_fd = target; + attr.attach_bpf_fd = prog_fd; + attr.attach_type = type; + attr.attach_flags = flags; + attr.relative_fd = relative;
instead of two lines above, have simple if/else
if (relative_if) {
attr.relative_id = relative_id;
attr.attach_flags = flags | BPF_F_ID;
} else {
attr.relative_fd = relative_fd;
attr.attach_flags = flags;
}This combined with the piece above seems very straightforward in terms of what is checked and what's passed into attr. WDYT?
Quoted 23 lines
tools/lib/bpf/bpf.c +13
-3+ attr.replace_bpf_fd =OPTS_GET (opts, replace_fd,0 ); + attr.expected_revision =OPTS_GET (opts, expected_revision,0 ); ret =sys_bpf (BPF_PROG_ATTACH , &attr, attr_sz);return libbpf_err_errno (ret); } -int bpf_prog_detach (int target_fd,enum bpf_attach_type type) +int bpf_prog_detach_opts (int prog_fd,int target, +enum bpf_attach_type type, +const struct bpf_prog_detach_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); + const size_t attr_sz = offsetofend(union bpf_attr, expected_revision); +__u32 relative_id, flags;union bpf_attr attr; -int ret; +int ret, relative; + +if (!OPTS_VALID (opts, bpf_prog_detach_opts)) +return libbpf_err (-EINVAL ); + + relative_id =OPTS_GET (opts, relative_id,0 );
tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 691
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+ relative =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + flags |=BPF_F_ID ; + }
see above, I think the same data flow simplification can be done
Quoted 15 lines
tools/lib/bpf/bpf.c +6
-2memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_type = type; + attr.target_fd = target; + attr.attach_bpf_fd = prog_fd; + attr.attach_type = type; + attr.attach_flags = flags; + attr.relative_fd = relative; + attr.expected_revision =OPTS_GET (opts, expected_revision,0 ); ret =sys_bpf (BPF_PROG_DETACH , &attr, attr_sz);return libbpf_err_errno (ret); }
[...]
tools/lib/bpf/libbpf.map LIBBPF_1.3.0 { · line 399
Quoted 9 lines
tools/lib/bpf/libbpf.map +1
-0@@ -396,4 +396,5 @@ LIBBPF_1.3.0 { global: bpf_obj_pin_opts; bpf_program__attach_netfilter; + bpf_prog_detach_opts;
I think it sorts before bpf_program__attach_netfilter?
Quoted 6 lines
tools/lib/bpf/libbpf.map +0
-1} LIBBPF_1.2.0; --2.34.1
Daniel Borkmann
maintainerRe: [PATCH bpf-next v4 3/8] libbpf: Add opts-based attach/detach/query API for tcx
Quoted 107 lines · Andrii Nakryiko
Quoted 12 lines · Daniel Borkmann
Extend libbpf attach opts and add a new detach opts API so this can be used to add/remove fd-based tcx BPF programs. The old-style
bpf_prog_detach()andbpf_prog_detach2()APIs are refactored to reuse the newbpf_prog_detach_opts()internally.The
bpf_prog_query_opts()API got extended to be able to handle the newlink_ids,link_attach_flagsand revision fields.For concrete usage examples, see the extensive selftests that have been developed as part of this series.
Quoted 8 lines
Signed-off-by Daniel Borkmann <[email protected]>--- tools/lib/bpf/bpf.c | 105 +++++++++++++++++++++++++-------------- tools/lib/bpf/bpf.h | 92 ++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.c | 12 +++-- tools/lib/bpf/libbpf.map | 1 + 4 files changed, 157 insertions(+), 53 deletions(-)Thanks for doc comments! Looks good, left a few nits with suggestions for simplifying code, but it's minor.
Acked-by Andrii Nakryiko <[email protected]>Quoted 22 lines
tools/lib/bpf/bpf.c +6
-5@@ -629,55 +629,87 @@ int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type,return bpf_prog_attach_opts (prog_fd, target_fd, type, &opts); } -int bpf_prog_attach_opts (int prog_fd,int target_fd, -enum bpf_attach_type type, -const struct bpf_prog_attach_opts *opts) +int bpf_prog_attach_opts (int prog_fd,int target, +enum bpf_attach_type type, +const struct bpf_prog_attach_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); + const size_t attr_sz = offsetofend(union bpf_attr, expected_revision); +__u32 relative_id, flags;union bpf_attr attr; -int ret; +int ret, relative;tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 650
Quoted 10 lines
tools/lib/bpf/bpf.c +7
-0if (!OPTS_VALID (opts, bpf_prog_attach_opts))return libbpf_err (-EINVAL ); + relative_id =OPTS_GET (opts, relative_id,0 ); + relative =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (relative >0 && relative_id) +return libbpf_err (-EINVAL );I left a comment in the next patch about this, I think it should be simple
if (relative_fd && relative_id) { /* bad */ }. But see the next patch for why.tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 654
Quoted 4 lines
tools/lib/bpf/bpf.c +4
-0+if (relative_id) { + relative = relative_id; + flags |=BPF_F_ID ; + }it's a bit hard to follow as written (to me at least). How about a slight variation that has less in-place state update
int
relative_fd,relative_id;
relative_fd=OPTS_GET(opts, relative_fd, 0);relative_id=OPTS_GET(opts, relative_id, 0);/* only one of fd or id can be specified */
if (relative_fd&&relative_id> 0)return libbpf_err(-EINVAL);... then see further below
tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 661
Quoted 12 lines
tools/lib/bpf/bpf.c +6
-5+memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_bpf_fd = prog_fd; - attr.attach_type = type; - attr.attach_flags =OPTS_GET (opts, flags,0 ); - attr.replace_bpf_fd =OPTS_GET (opts, replace_prog_fd,0 ); + attr.target_fd = target; + attr.attach_bpf_fd = prog_fd; + attr.attach_type = type; + attr.attach_flags = flags; + attr.relative_fd = relative;instead of two lines above, have simple if/else
if (relative_if) { attr.relative_id = relative_id; attr.attach_flags = flags | BPF_F_ID; } else { attr.relative_fd = relative_fd; attr.attach_flags = flags; }
Quoted 2 lines
This combined with the piece above seems very straightforward in terms of what is checked and what's passed into attr. WDYT?
All sgtm, I've implemented the suggestions locally for v5.
Quoted 54 lines
Quoted 23 lines
tools/lib/bpf/bpf.c +13
-3+ attr.replace_bpf_fd =OPTS_GET (opts, replace_fd,0 ); + attr.expected_revision =OPTS_GET (opts, expected_revision,0 ); ret =sys_bpf (BPF_PROG_ATTACH , &attr, attr_sz);return libbpf_err_errno (ret); } -int bpf_prog_detach (int target_fd,enum bpf_attach_type type) +int bpf_prog_detach_opts (int prog_fd,int target, +enum bpf_attach_type type, +const struct bpf_prog_detach_opts *opts) { - const size_t attr_sz = offsetofend(union bpf_attr, replace_bpf_fd); + const size_t attr_sz = offsetofend(union bpf_attr, expected_revision); +__u32 relative_id, flags;union bpf_attr attr; -int ret; +int ret, relative; + +if (!OPTS_VALID (opts, bpf_prog_detach_opts)) +return libbpf_err (-EINVAL ); + + relative_id =OPTS_GET (opts, relative_id,0 );tools/lib/bpf/bpf.c int bpf_prog_attach(int prog_fd, int target_fd, enum bpf_attach_type type, · line 691
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+ relative =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + flags |=BPF_F_ID ; + }see above, I think the same data flow simplification can be done
Quoted 15 lines
tools/lib/bpf/bpf.c +6
-2memset (&attr,0 , attr_sz); - attr.target_fd = target_fd; - attr.attach_type = type; + attr.target_fd = target; + attr.attach_bpf_fd = prog_fd; + attr.attach_type = type; + attr.attach_flags = flags; + attr.relative_fd = relative; + attr.expected_revision =OPTS_GET (opts, expected_revision,0 ); ret =sys_bpf (BPF_PROG_DETACH , &attr, attr_sz);return libbpf_err_errno (ret); }[...]
Quoted 11 lines
tools/lib/bpf/libbpf.map LIBBPF_1.3.0 { · line 399
Quoted 9 lines
tools/lib/bpf/libbpf.map +1
-0@@ -396,4 +396,5 @@ LIBBPF_1.3.0 { global: bpf_obj_pin_opts; bpf_program__attach_netfilter; + bpf_prog_detach_opts;I think it sorts before
bpf_program__attach_netfilter?
Yeap, also fixed.
Daniel Borkmann
maintainerPATCH v4 4/8 libbpf: Add link-based API for tcx
In reply to Daniel BorkmannImplement tcx BPF link support for libbpf.
The bpf_program__attach_fd() API has been refactored slightly in order to pass bpf_link_create_opts pointer as input.
A new bpf_program__attach_tcx() has been added on top of this which allows for passing all relevant data via extensible struct bpf_tcx_opts.
The program sections tcx/ingress and tcx/egress correspond to the hook locations for tc ingress and egress, respectively.
For concrete usage examples, see the extensive selftests that have been developed as part of this series.
--- tools/lib/bpf/bpf.c | 19 ++++++++++-- tools/lib/bpf/bpf.h | 5 ++++ tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.h | 16 +++++++++++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 92 insertions(+), 11 deletions(-)
tools/lib/bpf/bpf.c +17 -2
@@ -717,9 +717,9 @@ int bpf_link_create(int prog_fd, int target_fd, const struct bpf_link_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, link_create); -__u32 target_btf_id, iter_info_len; +__u32 target_btf_id, iter_info_len, relative_id; +int fd, err, relative;union bpf_attr attr; -int fd, err;if (!OPTS_VALID (opts, bpf_link_create_opts))return libbpf_err (-EINVAL );
@@ -781,6 +781,21 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, netfilter))return libbpf_err (-EINVAL );break ; +case BPF_TCX_INGRESS : +case BPF_TCX_EGRESS : + relative =OPTS_GET (opts, tcx.relative_fd,0 ); + relative_id =OPTS_GET (opts, tcx.relative_id,0 ); +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + attr.link_create.flags |=BPF_F_ID ; + } + attr.link_create.tcx.relative_fd = relative; + attr.link_create.tcx.expected_revision =OPTS_GET (opts, tcx.expected_revision,0 ); +if (!OPTS_ZEROED (opts, tcx)) +return libbpf_err (-EINVAL ); +break ;default :if (!OPTS_ZEROED (opts, flags))return libbpf_err (-EINVAL );
tools/lib/bpf/bpf.h +5 -0
@@ -401,6 +401,11 @@ struct bpf_link_create_opts {__s32 priority;__u32 flags; } netfilter; +struct { +__u32 relative_fd; +__u32 relative_id; +__u64 expected_revision; + } tcx; }; size_t :0; };
tools/lib/bpf/libbpf.c +53 -9
107 lines not on this page · raw message
tools/lib/bpf/libbpf.h +16 -0
@@ -733,6 +733,22 @@ LIBBPF_API struct bpf_link * bpf_program__attach_netfilter(const struct bpf_program *prog, const struct bpf_netfilter_opts *opts); +struct bpf_tcx_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +int ifindex; +__u32 flags; +__u32 relative_fd; +__u32 relative_id; +__u64 expected_revision; + size_t :0; +}; +#define bpf_tcx_opts__last_field expected_revision + +LIBBPF_API struct bpf_link * +bpf_program__attach_tcx (const struct bpf_program *prog, +const struct bpf_tcx_opts *opts); +struct bpf_map ; LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);
tools/lib/bpf/libbpf.map +1 -0
@@ -397,4 +397,5 @@ LIBBPF_1.3.0 { bpf_obj_pin_opts; bpf_program__attach_netfilter; bpf_prog_detach_opts; + bpf_program__attach_tcx; } LIBBPF_1.2.0;
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v4 4/8] libbpf: Add link-based API for tcx
Quoted 15 lines · Daniel Borkmann
Implement tcx BPF link support for libbpf.
The
bpf_program__attach_fd()API has been refactored slightly in order to passbpf_link_create_optspointer as input.A new
bpf_program__attach_tcx()has been added on top of this which allows for passing all relevant data via extensible structbpf_tcx_opts.The program sections tcx/ingress and tcx/egress correspond to the hook locations for tc ingress and egress, respectively.
For concrete usage examples, see the extensive selftests that have been developed as part of this series.
Quoted 9 lines
Signed-off-by Daniel Borkmann <[email protected]>--- tools/lib/bpf/bpf.c | 19 ++++++++++-- tools/lib/bpf/bpf.h | 5 ++++ tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.h | 16 +++++++++++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 92 insertions(+), 11 deletions(-)
Pretty minor nits, I think ifindex move to be mandatory argument is the most consequential, as it's an API. With that addressed, please add my ack for next rev
tools/lib/bpf/bpf.c int bpf_link_create(int prog_fd, int target_fd, · line 721
Quoted 11 lines
tools/lib/bpf/bpf.c +2
-1@@ -717,9 +717,9 @@ int bpf_link_create(int prog_fd, int target_fd, const struct bpf_link_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, link_create); -__u32 target_btf_id, iter_info_len; +__u32 target_btf_id, iter_info_len, relative_id; +int fd, err, relative;
nit: maybe make these new vars local to the TCX cases branch below?
Quoted 9 lines
tools/lib/bpf/bpf.c +0
-1union bpf_attr attr; -int fd, err;if (!OPTS_VALID (opts, bpf_link_create_opts))return libbpf_err (-EINVAL );@@ -781,6 +781,21 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, netfilter))return libbpf_err (-EINVAL );break ;
tools/lib/bpf/bpf.c int bpf_link_create(int prog_fd, int target_fd, · line 793
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+case BPF_TCX_INGRESS : +case BPF_TCX_EGRESS : + relative =OPTS_GET (opts, tcx.relative_fd,0 ); + relative_id =OPTS_GET (opts, tcx.relative_id,0 ); +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + attr.link_create.flags |=BPF_F_ID ; + }
Well, I have the same nit as in the previous patch, this "relative = relative_id" is both confusing because of naming asymmetry (no relative_fd throws me off), and also unnecessary updating of the state. link_create.flags |= BPF_F_ID is inevitable, but the rest can be more straightforward, IMO
Quoted 8 lines
tools/lib/bpf/bpf.c +5
-0+ attr.link_create.tcx.relative_fd = relative; + attr.link_create.tcx.expected_revision =OPTS_GET (opts, tcx.expected_revision,0 ); +if (!OPTS_ZEROED (opts, tcx)) +return libbpf_err (-EINVAL ); +break ;default :if (!OPTS_ZEROED (opts, flags))return libbpf_err (-EINVAL );
[...]
Quoted 12 lines
tools/lib/bpf/bpf.c +12
-0+struct bpf_link * +bpf_program__attach_tcx (const struct bpf_program *prog, +const struct bpf_tcx_opts *opts) +{ +LIBBPF_OPTS (bpf_link_create_opts, link_create_opts); +__u32 relative_id, flags; +int ifindex, relative_fd; + +if (!OPTS_VALID (opts, bpf_tcx_opts)) +return libbpf_err_ptr (-EINVAL ); + + relative_id =OPTS_GET (opts, relative_id,0 );
tools/lib/bpf/bpf.c
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+ relative_fd =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + ifindex =OPTS_GET (opts, ifindex,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (!ifindex) { +pr_warn ("prog '%s': target netdevice ifindex cannot be zero\n" , + prog->name); +return libbpf_err_ptr (-EINVAL ); + }
given ifindex is non-optional, then it makes more sense to have it as a mandatory argument between prog and opts in bpf_program__attach_tcx(), instead of as a field of an opts struct
Quoted 1 line
+ if (
relative_fd> 0 &&relative_id) {
this asymmetrical check is a bit distracting. And also, if someone specifies negative FD and positive ID, that's also a bad combo and we shouldn't just ignore invalid FD, right? So I'd have a nice and clean
if (relative_fd && relative_id) { /* bad */ }
tools/lib/bpf/bpf.c
Quoted 6 lines
tools/lib/bpf/bpf.c +6
-0+pr_warn ("prog '%s': relative_fd and relative_id cannot be set at the same time\n" , + prog->name); +return libbpf_err_ptr (-EINVAL ); + } +if (relative_id) + flags |=BPF_F_ID ;
I think bpf_link_create() will add this flag anyways, so can drop this adjustment logic here?
tools/lib/bpf/bpf.c
Quoted 8 lines
tools/lib/bpf/bpf.c +8
-0+ + link_create_opts.tcx.expected_revision =OPTS_GET (opts, expected_revision,0 ); + link_create_opts.tcx.relative_fd = relative_fd; + link_create_opts.tcx.relative_id = relative_id; + link_create_opts.flags = flags; + +/* target_fd/target_ifindex use the same field in LINK_CREATE */ +return bpf_program_attach_fd (prog, ifindex,"tc" , &link_create_opts);
s/tc/tcx/ ?
Quoted 26 lines
}
struct
bpf_link*bpf_program__attach_freplace(const structbpf_program*prog,tools/lib/bpf/bpf.c +6
-1@@ -11917,11 +11956,16 @@ struct bpf_link *bpf_program__attach_freplace(const struct bpf_program *prog, }if (target_fd) { +LIBBPF_OPTS (bpf_link_create_opts, target_opts); + btf_id =libbpf_find_prog_btf_id (attach_func_name, target_fd);if (btf_id <0 )return libbpf_err_ptr (btf_id); -return bpf_program__attach_fd (prog, target_fd, btf_id,"freplace" ); + target_opts.target_btf_id = btf_id; + +return bpf_program_attach_fd (prog, target_fd,"freplace" , + &target_opts); }else {/* no target, so use raw_tracepoint_open for compatibility * with old kernels tools/lib/bpf/libbpf.h +0
-0
tools/lib/bpf/libbpf.h LIBBPF_API struct bpf_link * · line 739
Quoted 7 lines
tools/lib/bpf/libbpf.h +4
-0@@ -733,6 +733,22 @@ LIBBPF_API struct bpf_link * bpf_program__attach_netfilter(const struct bpf_program *prog, const struct bpf_netfilter_opts *opts); +struct bpf_tcx_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +int ifindex;
is ifindex optional or it's expected to always be specified? If the latter, then I'd move ifindex out of opts and make it second arg of bpf_program__attach_tcx, between prog and opts
Quoted 20 lines
tools/lib/bpf/libbpf.h +12
-0+__u32 flags; +__u32 relative_fd; +__u32 relative_id; +__u64 expected_revision; + size_t :0; +}; +#define bpf_tcx_opts__last_field expected_revision + +LIBBPF_API struct bpf_link * +bpf_program__attach_tcx (const struct bpf_program *prog, +const struct bpf_tcx_opts *opts); +struct bpf_map ; LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);tools/lib/bpf/libbpf.map +0
-0
tools/lib/bpf/libbpf.map LIBBPF_1.3.0 { · line 400
Quoted 4 lines
tools/lib/bpf/libbpf.map +1
-0@@ -397,4 +397,5 @@ LIBBPF_1.3.0 { bpf_obj_pin_opts; bpf_program__attach_netfilter; bpf_prog_detach_opts; + bpf_program__attach_tcx;
heh, now we definitely screwed up sorting ;)
Quoted 5 lines
tools/lib/bpf/libbpf.map +0
-1} LIBBPF_1.2.0; --2.34.1
Daniel Borkmann
maintainerRe: [PATCH bpf-next v4 4/8] libbpf: Add link-based API for tcx
Quoted 101 lines · Andrii Nakryiko
Quoted 15 lines · Daniel Borkmann
Implement tcx BPF link support for libbpf.
The
bpf_program__attach_fd()API has been refactored slightly in order to passbpf_link_create_optspointer as input.A new
bpf_program__attach_tcx()has been added on top of this which allows for passing all relevant data via extensible structbpf_tcx_opts.The program sections tcx/ingress and tcx/egress correspond to the hook locations for tc ingress and egress, respectively.
For concrete usage examples, see the extensive selftests that have been developed as part of this series.
Quoted 9 lines
Signed-off-by Daniel Borkmann <[email protected]>--- tools/lib/bpf/bpf.c | 19 ++++++++++-- tools/lib/bpf/bpf.h | 5 ++++ tools/lib/bpf/libbpf.c | 62 ++++++++++++++++++++++++++++++++++------ tools/lib/bpf/libbpf.h | 16 +++++++++++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 92 insertions(+), 11 deletions(-)Pretty minor nits, I think ifindex move to be mandatory argument is the most consequential, as it's an API. With that addressed, please add my ack for next rev
Acked-by Andrii Nakryiko <[email protected]>tools/lib/bpf/bpf.c int bpf_link_create(int prog_fd, int target_fd, · line 721
Quoted 11 lines
tools/lib/bpf/bpf.c +2
-1@@ -717,9 +717,9 @@ int bpf_link_create(int prog_fd, int target_fd, const struct bpf_link_create_opts *opts) { const size_t attr_sz = offsetofend(union bpf_attr, link_create); -__u32 target_btf_id, iter_info_len; +__u32 target_btf_id, iter_info_len, relative_id; +int fd, err, relative;nit: maybe make these new vars local to the TCX cases branch below?
Quoted 9 lines
tools/lib/bpf/bpf.c +0
-1union bpf_attr attr; -int fd, err;if (!OPTS_VALID (opts, bpf_link_create_opts))return libbpf_err (-EINVAL );@@ -781,6 +781,21 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, netfilter))return libbpf_err (-EINVAL );break ;tools/lib/bpf/bpf.c int bpf_link_create(int prog_fd, int target_fd, · line 793
Quoted 10 lines
tools/lib/bpf/bpf.c +10
-0+case BPF_TCX_INGRESS : +case BPF_TCX_EGRESS : + relative =OPTS_GET (opts, tcx.relative_fd,0 ); + relative_id =OPTS_GET (opts, tcx.relative_id,0 ); +if (relative >0 && relative_id) +return libbpf_err (-EINVAL ); +if (relative_id) { + relative = relative_id; + attr.link_create.flags |=BPF_F_ID ; + }Well, I have the same nit as in the previous patch, this "relative =
relative_id" is both confusing because of naming asymmetry (norelative_fdthrows me off), and also unnecessary updating of the state. link_create.flags |=BPF_F_IDis inevitable, but the rest can be more straightforward, IMOQuoted 8 lines
tools/lib/bpf/bpf.c +5
-0+ attr.link_create.tcx.relative_fd = relative; + attr.link_create.tcx.expected_revision =OPTS_GET (opts, tcx.expected_revision,0 ); +if (!OPTS_ZEROED (opts, tcx)) +return libbpf_err (-EINVAL ); +break ;default :if (!OPTS_ZEROED (opts, flags))return libbpf_err (-EINVAL );[...]
Quoted 18 lines
tools/lib/bpf/bpf.c +18
-0+struct bpf_link * +bpf_program__attach_tcx (const struct bpf_program *prog, +const struct bpf_tcx_opts *opts) +{ +LIBBPF_OPTS (bpf_link_create_opts, link_create_opts); +__u32 relative_id, flags; +int ifindex, relative_fd; + +if (!OPTS_VALID (opts, bpf_tcx_opts)) +return libbpf_err_ptr (-EINVAL ); + + relative_id =OPTS_GET (opts, relative_id,0 ); + relative_fd =OPTS_GET (opts, relative_fd,0 ); + flags =OPTS_GET (opts, flags,0 ); + ifindex =OPTS_GET (opts, ifindex,0 ); + +/* validate we don't have unexpected combinations of non-zero fields */ +if (!ifindex) {
Quoted 8 lines
tools/lib/bpf/bpf.c
Quoted 4 lines
tools/lib/bpf/bpf.c +4
-0+pr_warn ("prog '%s': target netdevice ifindex cannot be zero\n" , + prog->name); +return libbpf_err_ptr (-EINVAL ); + }given ifindex is non-optional, then it makes more sense to have it as a mandatory argument between prog and opts in
bpf_program__attach_tcx(), instead of as a field of an opts struct
Agree, and it will also be more in line with bpf_program__attach_xdp() one which has ifindex as 2nd param too.
I also implemented the rest of the suggestions in here for v5, thanks!
Quoted 100 lines
Quoted 1 line
+ if (
relative_fd> 0 &&relative_id) {this asymmetrical check is a bit distracting. And also, if someone specifies negative FD and positive ID, that's also a bad combo and we shouldn't just ignore invalid FD, right? So I'd have a nice and clean
if (
relative_fd&&relative_id) { /* bad */ }tools/lib/bpf/bpf.c
Quoted 6 lines
tools/lib/bpf/bpf.c +6
-0+pr_warn ("prog '%s': relative_fd and relative_id cannot be set at the same time\n" , + prog->name); +return libbpf_err_ptr (-EINVAL ); + } +if (relative_id) + flags |=BPF_F_ID ;I think
bpf_link_create()will add this flag anyways, so can drop this adjustment logic here?tools/lib/bpf/bpf.c
Quoted 8 lines
tools/lib/bpf/bpf.c +8
-0+ + link_create_opts.tcx.expected_revision =OPTS_GET (opts, expected_revision,0 ); + link_create_opts.tcx.relative_fd = relative_fd; + link_create_opts.tcx.relative_id = relative_id; + link_create_opts.flags = flags; + +/* target_fd/target_ifindex use the same field in LINK_CREATE */ +return bpf_program_attach_fd (prog, ifindex,"tc" , &link_create_opts);s/tc/tcx/ ?
Quoted 26 lines
}struct
bpf_link*bpf_program__attach_freplace(const structbpf_program*prog,tools/lib/bpf/bpf.c +6
-1@@ -11917,11 +11956,16 @@ struct bpf_link *bpf_program__attach_freplace(const struct bpf_program *prog, }if (target_fd) { +LIBBPF_OPTS (bpf_link_create_opts, target_opts); + btf_id =libbpf_find_prog_btf_id (attach_func_name, target_fd);if (btf_id <0 )return libbpf_err_ptr (btf_id); -return bpf_program__attach_fd (prog, target_fd, btf_id,"freplace" ); + target_opts.target_btf_id = btf_id; + +return bpf_program_attach_fd (prog, target_fd,"freplace" , + &target_opts); }else {/* no target, so use raw_tracepoint_open for compatibility * with old kernels tools/lib/bpf/libbpf.h +0
-0tools/lib/bpf/libbpf.h LIBBPF_API struct bpf_link * · line 739
Quoted 7 lines
tools/lib/bpf/libbpf.h +4
-0@@ -733,6 +733,22 @@ LIBBPF_API struct bpf_link * bpf_program__attach_netfilter(const struct bpf_program *prog, const struct bpf_netfilter_opts *opts); +struct bpf_tcx_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; +int ifindex;is ifindex optional or it's expected to always be specified? If the latter, then I'd move ifindex out of opts and make it second arg of
bpf_program__attach_tcx, between prog and optsQuoted 20 lines
tools/lib/bpf/libbpf.h +12
-0+__u32 flags; +__u32 relative_fd; +__u32 relative_id; +__u64 expected_revision; + size_t :0; +}; +#define bpf_tcx_opts__last_field expected_revision + +LIBBPF_API struct bpf_link * +bpf_program__attach_tcx (const struct bpf_program *prog, +const struct bpf_tcx_opts *opts); +struct bpf_map ; LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);tools/lib/bpf/libbpf.map +0
-0tools/lib/bpf/libbpf.map LIBBPF_1.3.0 { · line 400
Quoted 4 lines
tools/lib/bpf/libbpf.map +1
-0@@ -397,4 +397,5 @@ LIBBPF_1.3.0 { bpf_obj_pin_opts; bpf_program__attach_netfilter; bpf_prog_detach_opts; + bpf_program__attach_tcx;heh, now we definitely screwed up sorting ;)
Quoted 5 lines
tools/lib/bpf/libbpf.map +0
-1} LIBBPF_1.2.0; --2.34.1
Daniel Borkmann
maintainerPATCH v4 5/8 libbpf: Add helper macro to clear opts structs
In reply to Daniel BorkmannAdd a small and generic LIBBPF_OPTS_CLEAR() helper macros which clears an opts structure and reinitializes its .sz member to place the structure size. I found this very useful when developing selftests, but it is also generic enough as a macro next to the existing LIBBPF_OPTS() which hides the .sz initialization, too.
--- tools/lib/bpf/libbpf_common.h | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/lib/bpf/libbpf_common.h +11 -0
@@ -70,4 +70,15 @@ }; \ }) +/* Helper macro to clear a libbpf options struct +* +* Small helper macro to reset all fields and to reinitialize the common +* structure size member. +*/ +#define LIBBPF_OPTS_CLEAR (NAME ) \ + do { \ + memset(&NAME, 0, sizeof(NAME)); \ + NAME.sz = sizeof(NAME); \ + } while (0) +#endif /* __LIBBPF_LIBBPF_COMMON_H */
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v4 5/8] libbpf: Add helper macro to clear opts structs
Quoted 21 lines · Daniel Borkmann
Add a small and generic
LIBBPF_OPTS_CLEAR()helper macros which clears an opts structure and reinitializes its .sz member to place the structure size. I found this very useful when developing selftests, but it is also generic enough as a macro next to the existingLIBBPF_OPTS()which hides the .sz initialization, too.Signed-off-by Daniel Borkmann <[email protected]>--- tools/lib/bpf/libbpf_common.h | 11 +++++++++++ 1 file changed, 11 insertions(+)tools/lib/bpf/libbpf_common.h +1
-0@@ -70,4 +70,15 @@ }; \ }) +/* Helper macro to clear a libbpf options struct
tools/lib/bpf/libbpf_common.h line 83
Quoted 10 lines
tools/lib/bpf/libbpf_common.h +10
-0+* +* Small helper macro to reset all fields and to reinitialize the common +* structure size member. +*/ +#define LIBBPF_OPTS_CLEAR (NAME ) \ + do { \ + memset(&NAME, 0, sizeof(NAME)); \ + NAME.sz = sizeof(NAME); \ + } while (0) +
This is fine, but I think you can go a half-step further and have something even more universal and useful. Something like this:
#define LIBBPF_OPTS_RESET(NAME, ...)
do {
memset(&NAME, 0, sizeof(NAME));
NAME = (typeof(NAME)) {
.sz = sizeof(struct TYPE),
__VA_ARGS__
};
while (0);I actually haven't tried if that typeof() trick works, but I hope it does :)
Then your LIBBPF_OPTS_CLEAR() is just LIBBPF_OPTS_RESET(x). But you can also re-initialize:
LIBBPF_OPTS_RESET(x, .flags = 123, .prog_fd = 456);
It's more in line with LIBBPF_OPTS() itself in capabilities, except it works on existing variable.
Quoted 4 lines
tools/lib/bpf/libbpf_common.h +0
-1#endif /* __LIBBPF_LIBBPF_COMMON_H */ --2.34.1
Daniel Borkmann
maintainerRe: [PATCH bpf-next v4 5/8] libbpf: Add helper macro to clear opts structs
Quoted 46 lines · Andrii Nakryiko
Quoted 21 lines · Daniel Borkmann
Add a small and generic
LIBBPF_OPTS_CLEAR()helper macros which clears an opts structure and reinitializes its .sz member to place the structure size. I found this very useful when developing selftests, but it is also generic enough as a macro next to the existingLIBBPF_OPTS()which hides the .sz initialization, too.Signed-off-by Daniel Borkmann <[email protected]>--- tools/lib/bpf/libbpf_common.h | 11 +++++++++++ 1 file changed, 11 insertions(+)tools/lib/bpf/libbpf_common.h +1
-0@@ -70,4 +70,15 @@ }; \ }) +/* Helper macro to clear a libbpf options struct tools/lib/bpf/libbpf_common.h line 83
Quoted 10 lines
tools/lib/bpf/libbpf_common.h +10
-0+* +* Small helper macro to reset all fields and to reinitialize the common +* structure size member. +*/ +#define LIBBPF_OPTS_CLEAR (NAME ) \ + do { \ + memset(&NAME, 0, sizeof(NAME)); \ + NAME.sz = sizeof(NAME); \ + } while (0) +This is fine, but I think you can go a half-step further and have something even more universal and useful. Something like this:
#define LIBBPF_OPTS_RESET(NAME, ...) do { memset(&NAME, 0, sizeof(NAME)); NAME = (typeof(NAME)) { .sz = sizeof(struct TYPE), __VA_ARGS__ }; while (0);
Quoted 1 line
I actually haven't tried if that
typeof()trick works, but I hope it does :)
It does, I've used this in BPF code for Cilium, too. ;)
Quoted 7 lines
Then your
LIBBPF_OPTS_CLEAR()is justLIBBPF_OPTS_RESET(x). But you can also re-initialize:
LIBBPF_OPTS_RESET(x, .flags = 123, .prog_fd = 456);It's more in line with
LIBBPF_OPTS()itself in capabilities, except it works on existing variable.
Agree, changed into ...
/* Helper macro to clear and optionally reinitialize libbpf options struct
*
* Small helper macro to reset all fields and to reinitialize the common
* structure size member. Values provided by users in struct initializer-
* syntax as varargs can be provided as well to reinitialize options struct
* specific members.
*/
#define LIBBPF_OPTS_RESET(NAME, ...) \
do { \
memset(&NAME, 0, sizeof(NAME)); \
NAME = (typeof(NAME)) { \
.sz = sizeof(NAME), \
__VA_ARGS__ \
}; \
} while (0)... and updated all the test cases.
Thanks,
Daniel
Daniel Borkmann
maintainerPATCH v4 6/8 bpftool: Extend net dump with tcx progs
In reply to Daniel BorkmannAdd support to dump fd-based attach types via bpftool. This includes both the tc BPF link and attach ops programs. Dumped information contain the attach location, function entry name, program ID and link ID when applicable.
Example with tc BPF link:
# ./bpftool net xdp:
tc:
bond0(4) tcx/ingress cil_from_netdev prog id 784 link id 10 bond0(4) tcx/egress cil_to_netdev prog id 804 link id 11
flow_dissector:
netfilter:
Example with tc BPF attach ops:
# ./bpftool net xdp:
tc:
bond0(4) tcx/ingress cil_from_netdev prog id 654 bond0(4) tcx/egress cil_to_netdev prog id 672
flow_dissector:
netfilter:
--- tools/bpf/bpftool/net.c | 86 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 82 insertions(+), 4 deletions(-)
tools/bpf/bpftool/net.c +82 -4
@@ -76,6 +76,11 @@ static const char * const attach_type_strings[] = { [NET_ATTACH_TYPE_XDP_OFFLOAD] = "xdpoffload", }; +static const char *const attach_loc_strings[] = { + [BPF_TCX_INGRESS ] ="tcx/ingress" , + [BPF_TCX_EGRESS ] ="tcx/egress" , +}; +const size_t net_attach_type_size =ARRAY_SIZE (attach_type_strings);static enum net_attach_type parse_attach_type (const char *str)
@@ -422,8 +427,80 @@ static int dump_filter_nlmsg(void *cookie, void *msg, struct nlattr **tb) filter_info->devname, filter_info->ifindex); } -static int show_dev_tc_bpf (int sock,unsigned int nl_pid, -struct ip_devname_ifindex *dev) +static const char *flags_strings (__u32 flags) +{ +return json_output ?"none" :"" ; +} + +static int __show_dev_tc_bpf_name (__u32 id,char *name,size_t len) +{ +struct bpf_prog_info info = {}; +__u32 ilen =sizeof (info); +int fd, ret; + + fd =bpf_prog_get_fd_by_id (id); +if (fd <0 ) +return fd; + ret =bpf_obj_get_info_by_fd (fd, &info, &ilen); +if (ret <0 ) + goto out; + ret = -ENOENT ; +if (info.name[0 ]) { +get_prog_full_name (&info, fd, name, len); + ret =0 ; + } +out: +close (fd); +return ret; +} + +static void __show_dev_tc_bpf (const struct ip_devname_ifindex *dev, +const enum bpf_attach_type loc) +{ +__u32 prog_flags[64 ] = {}, link_flags[64 ] = {}, i; +__u32 prog_ids[64 ] = {}, link_ids[64 ] = {}; +LIBBPF_OPTS (bpf_prog_query_opts, optq); +char prog_name[MAX_PROG_FULL_NAME ]; +int ret; + + optq.prog_ids = prog_ids; + optq.prog_attach_flags = prog_flags; + optq.link_ids = link_ids; + optq.link_attach_flags = link_flags; + optq.count =ARRAY_SIZE (prog_ids); + + ret =bpf_prog_query_opts (dev->ifindex, loc, &optq); +if (ret) +return ; +for (i =0 ; i < optq.count; i++) { +NET_START_OBJECT ; +NET_DUMP_STR ("devname" ,"%s" , dev->devname); +NET_DUMP_UINT ("ifindex" ,"(%u)" , dev->ifindex); +NET_DUMP_STR ("kind" ," %s" , attach_loc_strings[loc]); + ret =__show_dev_tc_bpf_name (prog_ids[i], prog_name, +sizeof (prog_name)); +if (!ret) +NET_DUMP_STR ("name" ," %s" , prog_name); +NET_DUMP_UINT ("prog_id" ," prog id %u" , prog_ids[i]); +if (prog_flags[i]) +NET_DUMP_STR ("prog_flags" ,"%s" ,flags_strings (prog_flags[i])); +if (link_ids[i]) +NET_DUMP_UINT ("link_id" ," link id %u" , + link_ids[i]); +if (link_flags[i]) +NET_DUMP_STR ("link_flags" ,"%s" ,flags_strings (link_flags[i])); +NET_END_OBJECT_FINAL ; + } +} + +static void show_dev_tc_bpf (struct ip_devname_ifindex *dev) +{ +__show_dev_tc_bpf (dev,BPF_TCX_INGRESS ); +__show_dev_tc_bpf (dev,BPF_TCX_EGRESS ); +} + +static int show_dev_tc_bpf_classic (int sock,unsigned int nl_pid, +struct ip_devname_ifindex *dev) {struct bpf_filter_t filter_info;struct bpf_tcinfo_t tcinfo;
@@ -790,8 +867,9 @@ static int do_show(int argc, char **argv)if (!ret) {NET_START_ARRAY ("tc" ,"%s:\n" );for (i =0 ; i < dev_array.used_len; i++) { - ret =show_dev_tc_bpf (sock, nl_pid, - &dev_array.devices[i]); +show_dev_tc_bpf (&dev_array.devices[i]); + ret =show_dev_tc_bpf_classic (sock, nl_pid, + &dev_array.devices[i]);if (ret)break ; }
Signature · 1 line
2.34.1
Re: [PATCH bpf-next v4 6/8] bpftool: Extend net dump with tcx progs
2023-07-10 22:12 UTC+0200 ~ Daniel Borkmann <[email protected]>
Quoted 26 lines
Add support to dump fd-based attach types via bpftool. This includes both the tc BPF link and attach ops programs. Dumped information contain the attach location, function entry name, program ID and link ID when applicable.
Example with tc BPF link:
# ./bpftool net xdp:tc:
bond0(4) tcx/ingresscil_from_netdevprog id 784 link id 10 bond0(4) tcx/egresscil_to_netdevprog id 804 link id 11
flow_dissector:netfilter:
Example with tc BPF attach ops:
# ./bpftool net xdp:tc:
bond0(4) tcx/ingresscil_from_netdevprog id 654 bond0(4) tcx/egresscil_to_netdevprog id 672
Quoted 5 lines
flow_dissector:netfilter:
Signed-off-by Daniel Borkmann <[email protected]>
Thank you!
If you respin, would you mind updating the docs please (Documentation/bpftool-net.rst), I realise it says that "bpftool net" only dumps for tc and XDP, but that's not true any more since we have the flow dissector, netfilter programs, and now tcx. The examples are out-of-date too, but updating them doesn't have to be part of this PR.
Quoted 71 lines
--- tools/bpf/bpftool/net.c | 86 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 82 insertions(+), 4 deletions(-)tools/bpf/bpftool/net.c +50
-2@@ -76,6 +76,11 @@ static const char * const attach_type_strings[] = { [NET_ATTACH_TYPE_XDP_OFFLOAD] = "xdpoffload", }; +static const char *const attach_loc_strings[] = { + [BPF_TCX_INGRESS ] ="tcx/ingress" , + [BPF_TCX_EGRESS ] ="tcx/egress" , +}; +const size_t net_attach_type_size =ARRAY_SIZE (attach_type_strings);static enum net_attach_type parse_attach_type (const char *str)@@ -422,8 +427,80 @@ static int dump_filter_nlmsg(void *cookie, void *msg, struct nlattr **tb) filter_info->devname, filter_info->ifindex); } -static int show_dev_tc_bpf (int sock,unsigned int nl_pid, -struct ip_devname_ifindex *dev) +static const char *flags_strings (__u32 flags) +{ +return json_output ?"none" :"" ; +} + +static int __show_dev_tc_bpf_name (__u32 id,char *name,size_t len) +{ +struct bpf_prog_info info = {}; +__u32 ilen =sizeof (info); +int fd, ret; + + fd =bpf_prog_get_fd_by_id (id); +if (fd <0 ) +return fd; + ret =bpf_obj_get_info_by_fd (fd, &info, &ilen); +if (ret <0 ) + goto out; + ret = -ENOENT ; +if (info.name[0 ]) { +get_prog_full_name (&info, fd, name, len); + ret =0 ; + } +out: +close (fd); +return ret; +} + +static void __show_dev_tc_bpf (const struct ip_devname_ifindex *dev, +const enum bpf_attach_type loc) +{ +__u32 prog_flags[64 ] = {}, link_flags[64 ] = {}, i; +__u32 prog_ids[64 ] = {}, link_ids[64 ] = {}; +LIBBPF_OPTS (bpf_prog_query_opts, optq); +char prog_name[MAX_PROG_FULL_NAME ]; +int ret; + + optq.prog_ids = prog_ids; + optq.prog_attach_flags = prog_flags; + optq.link_ids = link_ids; + optq.link_attach_flags = link_flags; + optq.count =ARRAY_SIZE (prog_ids); + + ret =bpf_prog_query_opts (dev->ifindex, loc, &optq); +if (ret) +return ;
tools/bpf/bpftool/net.c static int dump_filter_nlmsg(void *cookie, void *msg, struct nlattr **tb) · line 484
Quoted 10 lines
tools/bpf/bpftool/net.c +10
-0+for (i =0 ; i < optq.count; i++) { +NET_START_OBJECT ; +NET_DUMP_STR ("devname" ,"%s" , dev->devname); +NET_DUMP_UINT ("ifindex" ,"(%u)" , dev->ifindex); +NET_DUMP_STR ("kind" ," %s" , attach_loc_strings[loc]); + ret =__show_dev_tc_bpf_name (prog_ids[i], prog_name, +sizeof (prog_name)); +if (!ret) +NET_DUMP_STR ("name" ," %s" , prog_name); +NET_DUMP_UINT ("prog_id" ," prog id %u" , prog_ids[i]);
I was unsure at first about having two words for "prog id", or "link id" below (we use "prog_id" for netfilter, for example), but I see it leaves you the opportunity to append the flags, if any, without additional keywords so... why not.
Quoted 34 lines
tools/bpf/bpftool/net.c +22
-2+if (prog_flags[i]) +NET_DUMP_STR ("prog_flags" ,"%s" ,flags_strings (prog_flags[i])); +if (link_ids[i]) +NET_DUMP_UINT ("link_id" ," link id %u" , + link_ids[i]); +if (link_flags[i]) +NET_DUMP_STR ("link_flags" ,"%s" ,flags_strings (link_flags[i])); +NET_END_OBJECT_FINAL ; + } +} + +static void show_dev_tc_bpf (struct ip_devname_ifindex *dev) +{ +__show_dev_tc_bpf (dev,BPF_TCX_INGRESS ); +__show_dev_tc_bpf (dev,BPF_TCX_EGRESS ); +} + +static int show_dev_tc_bpf_classic (int sock,unsigned int nl_pid, +struct ip_devname_ifindex *dev) {struct bpf_filter_t filter_info;struct bpf_tcinfo_t tcinfo;@@ -790,8 +867,9 @@ static int do_show(int argc, char **argv)if (!ret) {NET_START_ARRAY ("tc" ,"%s:\n" );for (i =0 ; i < dev_array.used_len; i++) { - ret =show_dev_tc_bpf (sock, nl_pid, - &dev_array.devices[i]); +show_dev_tc_bpf (&dev_array.devices[i]); + ret =show_dev_tc_bpf_classic (sock, nl_pid, + &dev_array.devices[i]);if (ret)break ; }
Daniel Borkmann
maintainerRe: [PATCH bpf-next v4 6/8] bpftool: Extend net dump with tcx progs
Quoted 35 lines · Quentin Monnet
2023-07-10 22:12 UTC+0200 ~ Daniel Borkmann <[email protected]>
Quoted 26 lines
Add support to dump fd-based attach types via bpftool. This includes both the tc BPF link and attach ops programs. Dumped information contain the attach location, function entry name, program ID and link ID when applicable.
Example with tc BPF link:
# ./bpftool net xdp:tc:
bond0(4) tcx/ingresscil_from_netdevprog id 784 link id 10 bond0(4) tcx/egresscil_to_netdevprog id 804 link id 11
flow_dissector:netfilter:
Example with tc BPF attach ops:
# ./bpftool net xdp:tc:
bond0(4) tcx/ingresscil_from_netdevprog id 654 bond0(4) tcx/egresscil_to_netdevprog id 672Quoted 5 lines
flow_dissector:netfilter:
Signed-off-by Daniel Borkmann <[email protected]>Reviewed-by Quentin Monnet <[email protected]>
Quoted 7 lines
Thank you!
If you respin, would you mind updating the docs please (
Documentation/bpftool-net.rst), I realise it says that "bpftool net" only dumps for tc and XDP, but that's not true any more since we have the flow dissector, netfilter programs, and now tcx. The examples are out-of-date too, but updating them doesn't have to be part of this PR.
Good point, I updated the docs and help usage to reflect that.
Quoted 77 lines
Quoted 77 lines
tools/bpf/bpftool/net.c | 86 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 82 insertions(+), 4 deletions(-)tools/bpf/bpftool/net.c +57
-2@@ -76,6 +76,11 @@ static const char * const attach_type_strings[] = { [NET_ATTACH_TYPE_XDP_OFFLOAD] = "xdpoffload", }; +static const char *const attach_loc_strings[] = { + [BPF_TCX_INGRESS ] ="tcx/ingress" , + [BPF_TCX_EGRESS ] ="tcx/egress" , +}; +const size_t net_attach_type_size =ARRAY_SIZE (attach_type_strings);static enum net_attach_type parse_attach_type (const char *str)@@ -422,8 +427,80 @@ static int dump_filter_nlmsg(void *cookie, void *msg, struct nlattr **tb) filter_info->devname, filter_info->ifindex); } -static int show_dev_tc_bpf (int sock,unsigned int nl_pid, -struct ip_devname_ifindex *dev) +static const char *flags_strings (__u32 flags) +{ +return json_output ?"none" :"" ; +} + +static int __show_dev_tc_bpf_name (__u32 id,char *name,size_t len) +{ +struct bpf_prog_info info = {}; +__u32 ilen =sizeof (info); +int fd, ret; + + fd =bpf_prog_get_fd_by_id (id); +if (fd <0 ) +return fd; + ret =bpf_obj_get_info_by_fd (fd, &info, &ilen); +if (ret <0 ) + goto out; + ret = -ENOENT ; +if (info.name[0 ]) { +get_prog_full_name (&info, fd, name, len); + ret =0 ; + } +out: +close (fd); +return ret; +} + +static void __show_dev_tc_bpf (const struct ip_devname_ifindex *dev, +const enum bpf_attach_type loc) +{ +__u32 prog_flags[64 ] = {}, link_flags[64 ] = {}, i; +__u32 prog_ids[64 ] = {}, link_ids[64 ] = {}; +LIBBPF_OPTS (bpf_prog_query_opts, optq); +char prog_name[MAX_PROG_FULL_NAME ]; +int ret; + + optq.prog_ids = prog_ids; + optq.prog_attach_flags = prog_flags; + optq.link_ids = link_ids; + optq.link_attach_flags = link_flags; + optq.count =ARRAY_SIZE (prog_ids); + + ret =bpf_prog_query_opts (dev->ifindex, loc, &optq); +if (ret) +return ; +for (i =0 ; i < optq.count; i++) { +NET_START_OBJECT ; +NET_DUMP_STR ("devname" ,"%s" , dev->devname); +NET_DUMP_UINT ("ifindex" ,"(%u)" , dev->ifindex); +NET_DUMP_STR ("kind" ," %s" , attach_loc_strings[loc]); + ret =__show_dev_tc_bpf_name (prog_ids[i], prog_name, +sizeof (prog_name));
Quoted 8 lines
tools/bpf/bpftool/net.c static int dump_filter_nlmsg(void *cookie, void *msg, struct nlattr **tb) · line 484
Quoted 3 lines
tools/bpf/bpftool/net.c +3
-0+if (!ret) +NET_DUMP_STR ("name" ," %s" , prog_name); +NET_DUMP_UINT ("prog_id" ," prog id %u" , prog_ids[i]);I was unsure at first about having two words for "prog id", or "link id" below (we use "
prog_id" for netfilter, for example), but I see it leaves you the opportunity to append the flags, if any, without additional keywords so... why not.
Ok, I'll change it into prog_id, link_id for consistency for the human readable output.
And some like flow dissector just dump 'id'. After sync with Quentin, I tracked this in [0] to more streamline the net dump output for other types.
[0] https://github.com/libbpf/bpftool/issues/106
Thanks,
Daniel
Daniel Borkmann
maintainerPATCH v4 7/8 selftests/bpf: Add mprog API tests for BPF tcx opts
In reply to Daniel BorkmannAdd a big batch of test coverage to assert all aspects of the tcx opts attach, detach and query API:
# ./vmtest.sh -- ./test_progs -t tc_opts [...] #238 tc_opts_after:OK #239 tc_opts_append:OK #240 tc_opts_basic:OK #241 tc_opts_before:OK #242 tc_opts_chain_classic:OK #243 tc_opts_demixed:OK #244 tc_opts_detach:OK #245 tc_opts_detach_after:OK #246 tc_opts_detach_before:OK #247 tc_opts_dev_cleanup:OK #248 tc_opts_invalid:OK #249 tc_opts_mixed:OK #250 tc_opts_prepend:OK #251 tc_opts_replace:OK #252 tc_opts_revision:OK Summary: 15/0 PASSED, 0 SKIPPED, 0 FAILED
--- .../selftests/bpf/prog_tests/tc_helpers.h | 72 + .../selftests/bpf/prog_tests/tc_opts.c | 2182 +++++++++++++++++ .../selftests/bpf/progs/test_tc_link.c | 40 + 3 files changed, 2294 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_helpers.h create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_opts.c create mode 100644 tools/testing/selftests/bpf/progs/test_tc_link.c
tools/testing/selftests/bpf/prog_tests/tc_helpers.h new file +72 -0
@@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2023 Isovalent */ +#ifndef TC_HELPERS +#define TC_HELPERS +#include <test_progs.h> + +static inline __u32 id_from_prog_fd (int fd) +{ +struct bpf_prog_info prog_info = {}; +__u32 prog_info_len =sizeof (prog_info); +int err; + + err =bpf_obj_get_info_by_fd (fd, &prog_info, &prog_info_len); +if (!ASSERT_OK (err,"id_from_prog_fd" )) +return 0 ; + +ASSERT_NEQ (prog_info.id,0 ,"prog_info.id" ); +return prog_info.id; +} + +static inline __u32 id_from_link_fd (int fd) +{ +struct bpf_link_info link_info = {}; +__u32 link_info_len =sizeof (link_info); +int err; + + err =bpf_link_get_info_by_fd (fd, &link_info, &link_info_len); +if (!ASSERT_OK (err,"id_from_link_fd" )) +return 0 ; + +ASSERT_NEQ (link_info.id,0 ,"link_info.id" ); +return link_info.id; +} + +static inline __u32 ifindex_from_link_fd (int fd) +{ +struct bpf_link_info link_info = {}; +__u32 link_info_len =sizeof (link_info); +int err; + + err =bpf_link_get_info_by_fd (fd, &link_info, &link_info_len); +if (!ASSERT_OK (err,"id_from_link_fd" )) +return 0 ; + +return link_info.tcx.ifindex; +} + +static inline void __assert_mprog_count (int target,int expected,bool miniq,int ifindex) +{ +__u32 count =0 , attach_flags =0 ; +int err; + + err =bpf_prog_query (ifindex, target,0 , &attach_flags, +NULL , &count); +ASSERT_EQ (count, expected,"count" ); +if (!expected && !miniq) +ASSERT_EQ (err, -ENOENT ,"prog_query" ); +else +ASSERT_EQ (err,0 ,"prog_query" ); +} + +static inline void assert_mprog_count (int target,int expected) +{ +__assert_mprog_count (target, expected, false, loopback); +} + +static inline void assert_mprog_count_ifindex (int ifindex,int target,int expected) +{ +__assert_mprog_count (target, expected, false, ifindex); +} + +#endif /* TC_HELPERS */
tools/testing/selftests/bpf/prog_tests/tc_opts.c new file +2182 -0
2,182 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_tc_link.c new file +40 -0
@@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Isovalent */ +#include <stdbool.h> +#include <linux/bpf.h> +#include <bpf/bpf_helpers.h> + +char LICENSE[] SEC("license") = "GPL"; + +bool seen_tc1; +bool seen_tc2; +bool seen_tc3; +bool seen_tc4; + +SEC("tc/ingress") +int tc1 (struct __sk_buff *skb) +{ + seen_tc1 = true; +return TCX_NEXT ; +} + +SEC("tc/egress") +int tc2 (struct __sk_buff *skb) +{ + seen_tc2 = true; +return TCX_NEXT ; +} + +SEC("tc/egress") +int tc3 (struct __sk_buff *skb) +{ + seen_tc3 = true; +return TCX_NEXT ; +} + +SEC("tc/egress") +int tc4 (struct __sk_buff *skb) +{ + seen_tc4 = true; +return TCX_NEXT ; +}
Signature · 1 line
2.34.1
Daniel Borkmann
maintainerPATCH v4 8/8 selftests/bpf: Add mprog API tests for BPF tcx links
In reply to Daniel BorkmannAdd a big batch of test coverage to assert all aspects of the tcx link API:
# ./vmtest.sh -- ./test_progs -t tc_links [...] #225 tc_links_after:OK #226 tc_links_append:OK #227 tc_links_basic:OK #228 tc_links_before:OK #229 tc_links_chain_classic:OK #230 tc_links_dev_cleanup:OK #231 tc_links_invalid:OK #232 tc_links_prepend:OK #233 tc_links_replace:OK #234 tc_links_revision:OK Summary: 10/0 PASSED, 0 SKIPPED, 0 FAILED
--- .../selftests/bpf/prog_tests/tc_links.c | 1604 +++++++++++++++++ 1 file changed, 1604 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/tc_links.c
tools/testing/selftests/bpf/prog_tests/tc_links.c new file +1604 -0
@@ -0,0 +1,1604 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023 Isovalent */ +#include <uapi/linux/if_link.h> +#include <net/if.h> +#include <test_progs.h> + +#define loopback 1 +#define ping_cmd "ping -q -c1 -w1 127.0.0.1 > /dev/null" + +#include "test_tc_link.skel.h" +#include "tc_helpers.h" + +void serial_test_tc_links_basic (void ) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 prog_ids[2 ], link_ids[2 ]; +__u32 pid1, pid2, lid1, lid2; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); + +assert_mprog_count (BPF_TCX_INGRESS ,0 ); +assert_mprog_count (BPF_TCX_EGRESS ,0 ); + +ASSERT_EQ (skel->bss->seen_tc1, false,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, false,"seen_tc2" ); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (BPF_TCX_INGRESS ,1 ); +assert_mprog_count (BPF_TCX_EGRESS ,0 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback,BPF_TCX_INGRESS , &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,1 ,"count" ); +ASSERT_EQ (optq.revision,2 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ],0 ,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ],0 ,"link_ids[1]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, false,"seen_tc2" ); + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); +ASSERT_NEQ (lid1, lid2,"link_ids_1_2" ); + +assert_mprog_count (BPF_TCX_INGRESS ,1 ); +assert_mprog_count (BPF_TCX_EGRESS ,1 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback,BPF_TCX_EGRESS , &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,1 ,"count" ); +ASSERT_EQ (optq.revision,2 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid2,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid2,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ],0 ,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ],0 ,"link_ids[1]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +cleanup: +test_tc_link__destroy (skel); + +assert_mprog_count (BPF_TCX_INGRESS ,0 ); +assert_mprog_count (BPF_TCX_EGRESS ,0 ); +} + +static void test_tc_links_before_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 prog_ids[5 ], link_ids[5 ]; +__u32 pid1, pid2, pid3, pid4; +__u32 lid1, lid2, lid3, lid4; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc3, target), +0 ,"tc3_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc4, target), +0 ,"tc4_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + pid4 =id_from_prog_fd (bpf_program__fd (skel->progs.tc4)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid3, pid4,"prog_ids_3_4" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); + +assert_mprog_count (target,2 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,3 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid2,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid2,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"link_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, false,"seen_tc4" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + optl.relative_fd =bpf_program__fd (skel->progs.tc2); + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc3 = link; + + lid3 =id_from_link_fd (bpf_link__fd (skel->links.tc3)); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_LINK ; + optl.relative_id = lid1; + + link =bpf_program__attach_tcx (skel->progs.tc4, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc4 = link; + + lid4 =id_from_link_fd (bpf_link__fd (skel->links.tc4)); + +assert_mprog_count (target,4 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,4 ,"count" ); +ASSERT_EQ (optq.revision,5 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid4,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid4,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid1,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid1,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ], pid3,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ], lid3,"link_ids[2]" ); +ASSERT_EQ (optq.prog_ids[3 ], pid2,"prog_ids[3]" ); +ASSERT_EQ (optq.link_ids[3 ], lid2,"link_ids[3]" ); +ASSERT_EQ (optq.prog_ids[4 ],0 ,"prog_ids[4]" ); +ASSERT_EQ (optq.link_ids[4 ],0 ,"link_ids[4]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, true,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, true,"seen_tc4" ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_before (void ) +{ +test_tc_links_before_target (BPF_TCX_INGRESS ); +test_tc_links_before_target (BPF_TCX_EGRESS ); +} + +static void test_tc_links_after_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 prog_ids[5 ], link_ids[5 ]; +__u32 pid1, pid2, pid3, pid4; +__u32 lid1, lid2, lid3, lid4; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc3, target), +0 ,"tc3_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc4, target), +0 ,"tc4_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + pid4 =id_from_prog_fd (bpf_program__fd (skel->progs.tc4)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid3, pid4,"prog_ids_3_4" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); + +assert_mprog_count (target,2 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,3 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid2,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid2,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"link_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, false,"seen_tc4" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc3 = link; + + lid3 =id_from_link_fd (bpf_link__fd (skel->links.tc3)); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER |BPF_F_LINK ; + optl.relative_fd =bpf_link__fd (skel->links.tc2); + + link =bpf_program__attach_tcx (skel->progs.tc4, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc4 = link; + + lid4 =id_from_link_fd (bpf_link__fd (skel->links.tc4)); + +assert_mprog_count (target,4 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,4 ,"count" ); +ASSERT_EQ (optq.revision,5 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid3,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid3,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ], pid2,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ], lid2,"link_ids[2]" ); +ASSERT_EQ (optq.prog_ids[3 ], pid4,"prog_ids[3]" ); +ASSERT_EQ (optq.link_ids[3 ], lid4,"link_ids[3]" ); +ASSERT_EQ (optq.prog_ids[4 ],0 ,"prog_ids[4]" ); +ASSERT_EQ (optq.link_ids[4 ],0 ,"link_ids[4]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, true,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, true,"seen_tc4" ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_after (void ) +{ +test_tc_links_after_target (BPF_TCX_INGRESS ); +test_tc_links_after_target (BPF_TCX_EGRESS ); +} + +static void test_tc_links_revision_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 prog_ids[3 ], link_ids[3 ]; +__u32 pid1, pid2, lid1, lid2; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); + +assert_mprog_count (target,0 ); + + optl.expected_revision =1 ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + + optl.expected_revision =1 ; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,1 ); + + optl.expected_revision =2 ; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); + +assert_mprog_count (target,2 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,3 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid2,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid2,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"prog_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_revision (void ) +{ +test_tc_links_revision_target (BPF_TCX_INGRESS ); +test_tc_links_revision_target (BPF_TCX_EGRESS ); +} + +static void test_tc_chain_classic (int target,bool chain_tc_old) +{ + LIBBPF_OPTS(bpf_tc_opts, tc_opts, .handle = 1, .priority = 1); + LIBBPF_OPTS(bpf_tc_hook, tc_hook, .ifindex = loopback); +bool hook_created = false, tc_attached = false; +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +__u32 pid1, pid2, pid3; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + +if (chain_tc_old) { + tc_hook.attach_point = target ==BPF_TCX_INGRESS ? +BPF_TC_INGRESS :BPF_TC_EGRESS ; + err =bpf_tc_hook_create (&tc_hook); +if (err ==0 ) + hook_created = true; + err = err == -EEXIST ?0 : err; +if (!ASSERT_OK (err,"bpf_tc_hook_create" )) + goto cleanup; + + tc_opts.prog_fd =bpf_program__fd (skel->progs.tc3); + err =bpf_tc_attach (&tc_hook, &tc_opts); +if (!ASSERT_OK (err,"bpf_tc_attach" )) + goto cleanup; + tc_attached = true; + } + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + +assert_mprog_count (target,2 ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, chain_tc_old,"seen_tc3" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + skel->bss->seen_tc3 = false; + + err =bpf_link__detach (skel->links.tc2); +if (!ASSERT_OK (err,"prog_detach" )) + goto cleanup; + +assert_mprog_count (target,1 ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, false,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, chain_tc_old,"seen_tc3" ); +cleanup: +if (tc_attached) { + tc_opts.flags = tc_opts.prog_fd = tc_opts.prog_id =0 ; + err =bpf_tc_detach (&tc_hook, &tc_opts); +ASSERT_OK (err,"bpf_tc_detach" ); + } +if (hook_created) { + tc_hook.attach_point =BPF_TC_INGRESS |BPF_TC_EGRESS ; +bpf_tc_hook_destroy (&tc_hook); + } +assert_mprog_count (target,1 ); +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_chain_classic (void ) +{ +test_tc_chain_classic (BPF_TCX_INGRESS , false); +test_tc_chain_classic (BPF_TCX_EGRESS , false); +test_tc_chain_classic (BPF_TCX_INGRESS , true); +test_tc_chain_classic (BPF_TCX_EGRESS , true); +} + +static void test_tc_links_replace_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 pid1, pid2, pid3, lid1, lid2; +__u32 prog_ids[4 ], link_ids[4 ]; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc3, target), +0 ,"tc3_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + + optl.expected_revision =1 ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + optl.relative_id = pid1; + optl.expected_revision =2 ; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); + +assert_mprog_count (target,2 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,3 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid2,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid2,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid1,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid1,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"link_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + skel->bss->seen_tc3 = false; + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_REPLACE ; + optl.relative_fd =bpf_program__fd (skel->progs.tc2); + optl.expected_revision =3 ; + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,2 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_REPLACE |BPF_F_LINK ; + optl.relative_fd =bpf_link__fd (skel->links.tc2); + optl.expected_revision =3 ; + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,2 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_REPLACE |BPF_F_LINK |BPF_F_AFTER ; + optl.relative_id = lid2; + optl.expected_revision =0 ; + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,2 ); + + err =bpf_link__update_program (skel->links.tc2, skel->progs.tc3); +if (!ASSERT_OK (err,"link_update" )) + goto cleanup; + +assert_mprog_count (target,2 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,4 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid3,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid2,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid1,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid1,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"link_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, false,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, true,"seen_tc3" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + skel->bss->seen_tc3 = false; + + err =bpf_link__detach (skel->links.tc2); +if (!ASSERT_OK (err,"link_detach" )) + goto cleanup; + +assert_mprog_count (target,1 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,1 ,"count" ); +ASSERT_EQ (optq.revision,5 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ],0 ,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ],0 ,"link_ids[1]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, false,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + skel->bss->seen_tc3 = false; + + err =bpf_link__update_program (skel->links.tc1, skel->progs.tc1); +if (!ASSERT_OK (err,"link_update_self" )) + goto cleanup; + +assert_mprog_count (target,1 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,1 ,"count" ); +ASSERT_EQ (optq.revision,5 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ],0 ,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ],0 ,"link_ids[1]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, false,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_replace (void ) +{ +test_tc_links_replace_target (BPF_TCX_INGRESS ); +test_tc_links_replace_target (BPF_TCX_EGRESS ); +} + +static void test_tc_links_invalid_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 pid1, pid2, lid1; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); + +assert_mprog_count (target,0 ); + + optl.flags =BPF_F_BEFORE |BPF_F_AFTER ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_ID ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER |BPF_F_ID ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_ID ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_LINK ; + optl.relative_fd =bpf_program__fd (skel->progs.tc2); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_LINK ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.relative_fd =bpf_program__fd (skel->progs.tc2); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_AFTER ; + optl.relative_fd =bpf_program__fd (skel->progs.tc2); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_ID ; + optl.relative_id = pid2; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_ID ; + optl.relative_id =42 ; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_LINK ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER |BPF_F_LINK ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,0 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER |BPF_F_LINK ; + optl.relative_fd =bpf_program__fd (skel->progs.tc1); + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_LINK |BPF_F_ID ; + optl.relative_id = ~0 ; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_LINK |BPF_F_ID ; + optl.relative_id = lid1; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_ID ; + optl.relative_id = pid1; + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_ERR_PTR (link,"link_attach_should_fail" )) { +bpf_link__destroy (link); + goto cleanup; + } +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE |BPF_F_LINK |BPF_F_ID ; + optl.relative_id = lid1; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + +assert_mprog_count (target,2 ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_invalid (void ) +{ +test_tc_links_invalid_target (BPF_TCX_INGRESS ); +test_tc_links_invalid_target (BPF_TCX_EGRESS ); +} + +static void test_tc_links_prepend_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 prog_ids[5 ], link_ids[5 ]; +__u32 pid1, pid2, pid3, pid4; +__u32 lid1, lid2, lid3, lid4; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc3, target), +0 ,"tc3_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc4, target), +0 ,"tc4_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + pid4 =id_from_prog_fd (bpf_program__fd (skel->progs.tc4)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid3, pid4,"prog_ids_3_4" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); + +assert_mprog_count (target,2 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,3 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid2,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid2,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid1,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid1,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"link_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, false,"seen_tc4" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc3 = link; + + lid3 =id_from_link_fd (bpf_link__fd (skel->links.tc3)); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_BEFORE ; + + link =bpf_program__attach_tcx (skel->progs.tc4, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc4 = link; + + lid4 =id_from_link_fd (bpf_link__fd (skel->links.tc4)); + +assert_mprog_count (target,4 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,4 ,"count" ); +ASSERT_EQ (optq.revision,5 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid4,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid4,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid3,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid3,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ], pid2,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ], lid2,"link_ids[2]" ); +ASSERT_EQ (optq.prog_ids[3 ], pid1,"prog_ids[3]" ); +ASSERT_EQ (optq.link_ids[3 ], lid1,"link_ids[3]" ); +ASSERT_EQ (optq.prog_ids[4 ],0 ,"prog_ids[4]" ); +ASSERT_EQ (optq.link_ids[4 ],0 ,"link_ids[4]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, true,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, true,"seen_tc4" ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_prepend (void ) +{ +test_tc_links_prepend_target (BPF_TCX_INGRESS ); +test_tc_links_prepend_target (BPF_TCX_EGRESS ); +} + +static void test_tc_links_append_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl, + .ifindex = loopback, + ); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 prog_ids[5 ], link_ids[5 ]; +__u32 pid1, pid2, pid3, pid4; +__u32 lid1, lid2, lid3, lid4; +struct test_tc_link *skel; +struct bpf_link *link; +int err; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc3, target), +0 ,"tc3_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc4, target), +0 ,"tc4_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + pid4 =id_from_prog_fd (bpf_program__fd (skel->progs.tc4)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid3, pid4,"prog_ids_3_4" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + + lid1 =id_from_link_fd (bpf_link__fd (skel->links.tc1)); + +assert_mprog_count (target,1 ); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER ; + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + + lid2 =id_from_link_fd (bpf_link__fd (skel->links.tc2)); + +assert_mprog_count (target,2 ); + + optq.prog_ids = prog_ids; + optq.link_ids = link_ids; + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,2 ,"count" ); +ASSERT_EQ (optq.revision,3 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid2,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid2,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ],0 ,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ],0 ,"link_ids[2]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, false,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, false,"seen_tc4" ); + + skel->bss->seen_tc1 = false; + skel->bss->seen_tc2 = false; + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER ; + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc3 = link; + + lid3 =id_from_link_fd (bpf_link__fd (skel->links.tc3)); + +LIBBPF_OPTS_CLEAR (optl); + optl.ifindex = loopback; + optl.flags =BPF_F_AFTER ; + + link =bpf_program__attach_tcx (skel->progs.tc4, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc4 = link; + + lid4 =id_from_link_fd (bpf_link__fd (skel->links.tc4)); + +assert_mprog_count (target,4 ); + +memset (prog_ids,0 ,sizeof (prog_ids)); +memset (link_ids,0 ,sizeof (link_ids)); + optq.count =ARRAY_SIZE (prog_ids); + + err =bpf_prog_query_opts (loopback, target, &optq); +if (!ASSERT_OK (err,"prog_query" )) + goto cleanup; + +ASSERT_EQ (optq.count,4 ,"count" ); +ASSERT_EQ (optq.revision,5 ,"revision" ); +ASSERT_EQ (optq.prog_ids[0 ], pid1,"prog_ids[0]" ); +ASSERT_EQ (optq.link_ids[0 ], lid1,"link_ids[0]" ); +ASSERT_EQ (optq.prog_ids[1 ], pid2,"prog_ids[1]" ); +ASSERT_EQ (optq.link_ids[1 ], lid2,"link_ids[1]" ); +ASSERT_EQ (optq.prog_ids[2 ], pid3,"prog_ids[2]" ); +ASSERT_EQ (optq.link_ids[2 ], lid3,"link_ids[2]" ); +ASSERT_EQ (optq.prog_ids[3 ], pid4,"prog_ids[3]" ); +ASSERT_EQ (optq.link_ids[3 ], lid4,"link_ids[3]" ); +ASSERT_EQ (optq.prog_ids[4 ],0 ,"prog_ids[4]" ); +ASSERT_EQ (optq.link_ids[4 ],0 ,"link_ids[4]" ); + +ASSERT_OK (system (ping_cmd), ping_cmd); + +ASSERT_EQ (skel->bss->seen_tc1, true,"seen_tc1" ); +ASSERT_EQ (skel->bss->seen_tc2, true,"seen_tc2" ); +ASSERT_EQ (skel->bss->seen_tc3, true,"seen_tc3" ); +ASSERT_EQ (skel->bss->seen_tc4, true,"seen_tc4" ); +cleanup: +test_tc_link__destroy (skel); +assert_mprog_count (target,0 ); +} + +void serial_test_tc_links_append (void ) +{ +test_tc_links_append_target (BPF_TCX_INGRESS ); +test_tc_links_append_target (BPF_TCX_EGRESS ); +} + +static void test_tc_links_dev_cleanup_target (int target) +{ +LIBBPF_OPTS (bpf_tcx_opts, optl); +LIBBPF_OPTS (bpf_prog_query_opts, optq); +__u32 pid1, pid2, pid3, pid4; +struct test_tc_link *skel; +struct bpf_link *link; +int err, ifindex; + +ASSERT_OK (system ("ip link add dev tcx_opts1 type veth peer name tcx_opts2" ),"add veth" ); + ifindex =if_nametoindex ("tcx_opts1" ); +ASSERT_NEQ (ifindex,0 ,"non_zero_ifindex" ); + optl.ifindex = ifindex; + + skel =test_tc_link__open (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc1, target), +0 ,"tc1_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc2, target), +0 ,"tc2_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc3, target), +0 ,"tc3_attach_type" ); +ASSERT_EQ (bpf_program__set_expected_attach_type (skel->progs.tc4, target), +0 ,"tc4_attach_type" ); + + err =test_tc_link__load (skel); +if (!ASSERT_OK (err,"skel_load" )) + goto cleanup; + + pid1 =id_from_prog_fd (bpf_program__fd (skel->progs.tc1)); + pid2 =id_from_prog_fd (bpf_program__fd (skel->progs.tc2)); + pid3 =id_from_prog_fd (bpf_program__fd (skel->progs.tc3)); + pid4 =id_from_prog_fd (bpf_program__fd (skel->progs.tc4)); + +ASSERT_NEQ (pid1, pid2,"prog_ids_1_2" ); +ASSERT_NEQ (pid3, pid4,"prog_ids_3_4" ); +ASSERT_NEQ (pid2, pid3,"prog_ids_2_3" ); + +assert_mprog_count (target,0 ); + + link =bpf_program__attach_tcx (skel->progs.tc1, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc1 = link; + +assert_mprog_count_ifindex (ifindex, target,1 ); + + link =bpf_program__attach_tcx (skel->progs.tc2, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc2 = link; + +assert_mprog_count_ifindex (ifindex, target,2 ); + + link =bpf_program__attach_tcx (skel->progs.tc3, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc3 = link; + +assert_mprog_count_ifindex (ifindex, target,3 ); + + link =bpf_program__attach_tcx (skel->progs.tc4, &optl); +if (!ASSERT_OK_PTR (link,"link_attach" )) + goto cleanup; + + skel->links.tc4 = link; + +assert_mprog_count_ifindex (ifindex, target,4 ); + +ASSERT_OK (system ("ip link del dev tcx_opts1" ),"del veth" ); +ASSERT_EQ (if_nametoindex ("tcx_opts1" ),0 ,"dev1_removed" ); +ASSERT_EQ (if_nametoindex ("tcx_opts2" ),0 ,"dev2_removed" ); + +ASSERT_EQ (ifindex_from_link_fd (bpf_link__fd (skel->links.tc1)),0 ,"tc1_ifindex" ); +ASSERT_EQ (ifindex_from_link_fd (bpf_link__fd (skel->links.tc2)),0 ,"tc2_ifindex" ); +ASSERT_EQ (ifindex_from_link_fd (bpf_link__fd (skel->links.tc3)),0 ,"tc3_ifindex" ); +ASSERT_EQ (ifindex_from_link_fd (bpf_link__fd (skel->links.tc4)),0 ,"tc4_ifindex" ); + +test_tc_link__destroy (skel); +return ; +cleanup: +test_tc_link__destroy (skel); + +ASSERT_OK (system ("ip link del dev tcx_opts1" ),"del veth" ); +ASSERT_EQ (if_nametoindex ("tcx_opts1" ),0 ,"dev1_removed" ); +ASSERT_EQ (if_nametoindex ("tcx_opts2" ),0 ,"dev2_removed" ); +} + +void serial_test_tc_links_dev_cleanup (void ) +{ +test_tc_links_dev_cleanup_target (BPF_TCX_INGRESS ); +test_tc_links_dev_cleanup_target (BPF_TCX_EGRESS ); +}
Signature · 1 line
2.34.1