Support defragmenting IPv(4|6) packets in BPF
Subsystems (7)
cover Support defragmenting IPv(4|6) packets in BPF
=== Context ===
In the context of a middlebox, fragmented packets are tricky to handle. The full 5-tuple of a packet is often only available in the first fragment which makes enforcing consistent policy difficult. There are really only two stateless options, neither of which are very nice:
1. Enforce policy on first fragment and accept all subsequent fragments. This works but may let in certain attacks or allow data exfiltration.
2. Enforce policy on first fragment and drop all subsequent fragments. This does not really work b/c some protocols may rely on fragmentation. For example, DNS may rely on oversized UDP packets for large responses.
So stateful tracking is the only sane option. RFC 8900 [0] calls this out as well in section 6.3:
Middleboxes [...] should process IP fragments in a manner that is consistent with [RFC0791] and [RFC8200]. In many cases, middleboxes must maintain state in order to achieve this goal.
=== BPF related bits ===
Policy has traditionally been enforced from XDP/TC hooks. Both hooks run before kernel reassembly facilities. However, with the new BPF_PROG_TYPE_NETFILTER, we can rather easily hook into existing netfilter reassembly infra.
The basic idea is we bump a refcnt on the netfilter defrag module and then run the bpf prog after the defrag module runs. This allows bpf progs to transparently see full, reassembled packets. The nice thing about this is that progs don't have to carry around logic to detect fragments.
=== Patchset details ===
There was an earlier attempt at providing defrag via kfuncs [1]. The feedback was that we could end up doing too much stuff in prog execution context (like sending ICMP error replies). However, I think there are still some outstanding discussion w.r.t. performance when it comes to netfilter vs the previous approach. I'll schedule some time during office hours for this.
Patches 1 & 2 are stolenfrom Florian. Hopefully he doesn't mind. There were some outstanding comments on the v2 [2] but it doesn't look like a v3 was ever submitted. I've addressed the comments and put them in this patchset cuz I needed them.
Finally, the new selftest seems to be a little flaky. I'm not quite sure why the server will fail to recvfrom() occassionaly. I'm fairly sure it's a timing related issue with creating veths. I'll keep debugging but I didn't want that to hold up discussion on this patchset.
[0]: https://datatracker.ietf.org/doc/html/rfc8900
[1]: https://lore.kernel.org/bpf/[email protected]/
[2]: https://lore.kernel.org/bpf/[email protected]/
Daniel Xu (7):
tools: libbpf: add netfilter link attach helper selftests/bpf: Add bpf_program__attach_netfilter helper test
netfilter: defrag: Add glue hooks for enabling/disabling defrag
netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link
bpf: selftests: Support not connecting client socket
bpf: selftests: Support custom type and proto for client sockets
bpf: selftests: Add defrag selftests
include/linux/netfilter.h | 12 + include/uapi/linux/bpf.h | 5 + net/ipv4/netfilter/nf_defrag_ipv4.c | 8 + net/ipv6/netfilter/nf_defrag_ipv6_hooks.c | 10 + net/netfilter/core.c | 6 + net/netfilter/nf_bpf_link.c | 108 ++++++- tools/include/uapi/linux/bpf.h | 5 + tools/lib/bpf/bpf.c | 8 + tools/lib/bpf/bpf.h | 6 + tools/lib/bpf/libbpf.c | 47 +++ tools/lib/bpf/libbpf.h | 15 + tools/lib/bpf/libbpf.map | 1 + tools/testing/selftests/bpf/Makefile | 4 +- .../selftests/bpf/generate_udp_fragments.py | 90 ++++++ .../selftests/bpf/ip_check_defrag_frags.h | 57 ++++ tools/testing/selftests/bpf/network_helpers.c | 26 +- tools/testing/selftests/bpf/network_helpers.h | 3 + .../bpf/prog_tests/ip_check_defrag.c | 282 ++++++++++++++++++ .../bpf/prog_tests/netfilter_basic.c | 78 +++++ .../selftests/bpf/progs/ip_check_defrag.c | 104 +++++++ .../bpf/progs/test_netfilter_link_attach.c | 14 + 21 files changed, 868 insertions(+), 21 deletions(-) create mode 100755 tools/testing/selftests/bpf/generate_udp_fragments.py create mode 100644 tools/testing/selftests/bpf/ip_check_defrag_frags.h create mode 100644 tools/testing/selftests/bpf/prog_tests/ip_check_defrag.c create mode 100644 tools/testing/selftests/bpf/prog_tests/netfilter_basic.c create mode 100644 tools/testing/selftests/bpf/progs/ip_check_defrag.c create mode 100644 tools/testing/selftests/bpf/progs/test_netfilter_link_attach.c
Signature · 1 line
2.40.1
PATCH 1/7 tools: libbpf: add netfilter link attach helper
Add new api function: bpf_program__attach_netfilter.
It takes a bpf program (netfilter type), and a pointer to a option struct that contains the desired attachment (protocol family, priority, hook location, ...).
It returns a pointer to a 'bpf_link' structure or NULL on error.
Next patch adds new netfilter_basic test that uses this function to attach a program to a few pf/hook/priority combinations.
--- tools/lib/bpf/bpf.c | 8 +++++++ tools/lib/bpf/bpf.h | 6 +++++ tools/lib/bpf/libbpf.c | 47 ++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 15 +++++++++++++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 77 insertions(+)
tools/lib/bpf/bpf.c +8 -0
@@ -741,6 +741,14 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, tracing))return libbpf_err (-EINVAL );break ; +case BPF_NETFILTER : + attr.link_create.netfilter.pf =OPTS_GET (opts, netfilter.pf,0 ); + attr.link_create.netfilter.hooknum =OPTS_GET (opts, netfilter.hooknum,0 ); + attr.link_create.netfilter.priority =OPTS_GET (opts, netfilter.priority,0 ); + attr.link_create.netfilter.flags =OPTS_GET (opts, netfilter.flags,0 ); +if (!OPTS_ZEROED (opts, netfilter)) +return libbpf_err (-EINVAL ); +break ;default :if (!OPTS_ZEROED (opts, flags))return libbpf_err (-EINVAL );
tools/lib/bpf/bpf.h +6 -0
@@ -349,6 +349,12 @@ struct bpf_link_create_opts {struct {__u64 cookie; } tracing; +struct { +__u32 pf; +__u32 hooknum; +__s32 priority; +__u32 flags; + } netfilter; }; size_t :0; };
tools/lib/bpf/libbpf.c +47 -0
53 lines not on this page · raw message
tools/lib/bpf/libbpf.h +15 -0
@@ -718,6 +718,21 @@ LIBBPF_API struct bpf_link * bpf_program__attach_freplace(const struct bpf_program *prog, int target_fd, const char *attach_func_name); +struct bpf_netfilter_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; + +__u32 pf; +__u32 hooknum; +__s32 priority; +__u32 flags; +}; +#define bpf_netfilter_opts__last_field flags + +LIBBPF_API struct bpf_link * +bpf_program__attach_netfilter (const struct bpf_program *prog, +const struct bpf_netfilter_opts *opts); +struct bpf_map ; LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);
tools/lib/bpf/libbpf.map +1 -0
@@ -395,4 +395,5 @@ LIBBPF_1.2.0 { LIBBPF_1.3.0 { global: bpf_obj_pin_opts; + bpf_program__attach_netfilter; } LIBBPF_1.2.0;
Signature · 1 line
2.40.1
Re: [PATCH bpf-next 1/7] tools: libbpf: add netfilter link attach helper
Quoted 65 lines · Daniel Xu
Add new api function:
bpf_program__attach_netfilter.It takes a bpf program (netfilter type), and a pointer to a option struct that contains the desired attachment (protocol family, priority, hook location, ...).
It returns a pointer to a '
bpf_link' structure or NULL on error.Next patch adds new
netfilter_basictest that uses this function to attach a program to a few pf/hook/priority combinations.Co-developed-by Florian Westphal <[email protected]>Signed-off-by Florian Westphal <[email protected]>Suggested-by Andrii Nakryiko <[email protected]>Signed-off-by Daniel Xu <[email protected]>--- tools/lib/bpf/bpf.c | 8 +++++++ tools/lib/bpf/bpf.h | 6 +++++ tools/lib/bpf/libbpf.c | 47 ++++++++++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.h | 15 +++++++++++++ tools/lib/bpf/libbpf.map | 1 + 5 files changed, 77 insertions(+)tools/lib/bpf/bpf.c +8
-0@@ -741,6 +741,14 @@ int bpf_link_create(int prog_fd, int target_fd,if (!OPTS_ZEROED (opts, tracing))return libbpf_err (-EINVAL );break ; +case BPF_NETFILTER : + attr.link_create.netfilter.pf =OPTS_GET (opts, netfilter.pf,0 ); + attr.link_create.netfilter.hooknum =OPTS_GET (opts, netfilter.hooknum,0 ); + attr.link_create.netfilter.priority =OPTS_GET (opts, netfilter.priority,0 ); + attr.link_create.netfilter.flags =OPTS_GET (opts, netfilter.flags,0 ); +if (!OPTS_ZEROED (opts, netfilter)) +return libbpf_err (-EINVAL ); +break ;default :if (!OPTS_ZEROED (opts, flags))return libbpf_err (-EINVAL );tools/lib/bpf/bpf.h +6
-0@@ -349,6 +349,12 @@ struct bpf_link_create_opts {struct {__u64 cookie; } tracing; +struct { +__u32 pf; +__u32 hooknum; +__s32 priority; +__u32 flags; + } netfilter; }; size_t :0; };tools/lib/bpf/libbpf.c +0
-0
tools/lib/bpf/libbpf.c static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_l · line 11817
Quoted 7 lines
tools/lib/bpf/libbpf.c +4
-0@@ -11811,6 +11811,53 @@ static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_lreturn libbpf_get_error (*link); } +struct bpf_link *bpf_program__attach_netfilter (const struct bpf_program *prog, +const struct bpf_netfilter_opts *opts) +{ +DECLARE_LIBBPF_OPTS (bpf_link_create_opts, link_create_opts);
nit: let's use shorter LIBBPF_OPTS() macro
Quoted 15 lines
tools/lib/bpf/libbpf.c +15
-0+struct bpf_link *link; +int prog_fd, link_fd; + +if (!OPTS_VALID (opts, bpf_netfilter_opts)) +return libbpf_err_ptr (-EINVAL ); + + link_create_opts.netfilter.pf =OPTS_GET (opts, pf,0 ); + link_create_opts.netfilter.hooknum =OPTS_GET (opts, hooknum,0 ); + link_create_opts.netfilter.priority =OPTS_GET (opts, priority,0 ); + link_create_opts.netfilter.flags =OPTS_GET (opts, flags,0 ); + + prog_fd =bpf_program__fd (prog); +if (prog_fd <0 ) { +pr_warn ("prog '%s': can't attach before loaded\n" , prog->name); +return libbpf_err_ptr (-EINVAL );
tools/lib/bpf/libbpf.c static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_l · line 11842
Quoted 10 lines
tools/lib/bpf/libbpf.c +10
-0+ } + + link =calloc (1 ,sizeof (*link)); +if (!link) +return libbpf_err_ptr (-ENOMEM ); + link->detach = &bpf_link__detach_fd; + + link_fd =bpf_link_create (prog_fd,0 ,BPF_NETFILTER , &link_create_opts); + + link->fd =ensure_good_fd (link_fd);
bpf_link_create() does ensure_good_fd() already, no need to do it here, just assign result directly
tools/lib/bpf/libbpf.c static int attach_iter(const struct bpf_program *prog, long cookie, struct bpf_l · line 11849
Quoted 7 lines
tools/lib/bpf/libbpf.c +7
-0+ +if (link->fd <0 ) { +char errmsg[STRERR_BUFSIZE ]; + + link_fd = -errno; +free (link); + pr_warn("prog '%s': failed to attach to pf:%d,hooknum:%d:prio:%d: %s\n",
comma before prio? but also how necessary is to emit all these? what if we add another argument to opts, would we add them here as well?
I'd just go with just "failed to attach netfilter" and keep it simple
Quoted 53 lines
tools/lib/bpf/libbpf.c +11
-0+ prog->name, +OPTS_GET (opts, pf,0 ), +OPTS_GET (opts, hooknum,0 ), +OPTS_GET (opts, priority,0 ), + libbpf_strerror_r(link_fd, errmsg, sizeof(errmsg))); +return libbpf_err_ptr (link_fd); + } + +return link; +} +struct bpf_link *bpf_program__attach (const struct bpf_program *prog) {struct bpf_link *link =NULL ;tools/lib/bpf/libbpf.h +15
-0@@ -718,6 +718,21 @@ LIBBPF_API struct bpf_link * bpf_program__attach_freplace(const struct bpf_program *prog, int target_fd, const char *attach_func_name); +struct bpf_netfilter_opts { +/* size of this struct, for forward/backward compatibility */ +size_t sz; + +__u32 pf; +__u32 hooknum; +__s32 priority; +__u32 flags; +}; +#define bpf_netfilter_opts__last_field flags + +LIBBPF_API struct bpf_link * +bpf_program__attach_netfilter (const struct bpf_program *prog, +const struct bpf_netfilter_opts *opts); +struct bpf_map ; LIBBPF_API struct bpf_link *bpf_map__attach_struct_ops(const struct bpf_map *map);tools/lib/bpf/libbpf.map +1
-1@@ -395,4 +395,5 @@ LIBBPF_1.2.0 { LIBBPF_1.3.0 { global: bpf_obj_pin_opts; + bpf_program__attach_netfilter; } LIBBPF_1.2.0; --2.40.1
PATCH 2/7 selftests/bpf: Add bpf_program__attach_netfilter helper test
In reply to Daniel XuCall bpf_program__attach_netfilter() with different protocol/hook/priority combinations.
Test fails if supposedly-illegal attachments work (e.g., bogus protocol family, illegal priority and so on) or if a should-work attachment fails.
--- .../bpf/prog_tests/netfilter_basic.c | 78 +++++++++++++++++++ .../bpf/progs/test_netfilter_link_attach.c | 14 ++++ 2 files changed, 92 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/netfilter_basic.c create mode 100644 tools/testing/selftests/bpf/progs/test_netfilter_link_attach.c
tools/testing/selftests/bpf/prog_tests/netfilter_basic.c new file +78 -0
@@ -0,0 +1,78 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#include <netinet/in.h> +#include <linux/netfilter.h> + +#include "test_progs.h" +#include "test_netfilter_link_attach.skel.h" + +struct nf_hook_options { +__u32 pf; +__u32 hooknum; +__s32 priority; +__u32 flags; + +bool expect_success; +}; + +struct nf_hook_options nf_hook_attach_tests[] = { + { }, + { .pf =NFPROTO_NUMPROTO , }, + { .pf =NFPROTO_IPV4 , .hooknum =42 , }, + { .pf =NFPROTO_IPV4 , .priority =INT_MIN }, + { .pf =NFPROTO_IPV4 , .priority =INT_MAX }, + { .pf =NFPROTO_IPV4 , .flags =UINT_MAX }, + + { .pf =NFPROTO_INET , .priority =1 , }, + + { .pf =NFPROTO_IPV4 , .priority =-10000 , .expect_success = true }, + { .pf =NFPROTO_IPV6 , .priority =10001 , .expect_success = true }, +}; + +void test_netfilter_basic (void ) +{ +struct test_netfilter_link_attach *skel; +LIBBPF_OPTS (bpf_netfilter_opts, opts); +struct bpf_program *prog; +int i; + + skel =test_netfilter_link_attach__open_and_load (); +if (!ASSERT_OK_PTR (skel,"test_netfilter_link_attach__open_and_load" )) + goto out; + + prog = skel->progs.nf_link_attach_test; + +for (i =0 ; i <ARRAY_SIZE (nf_hook_attach_tests); i++) { +struct bpf_link *link; + +#define X (opts, m, i) opts.m = nf_hook_attach_tests[(i)].m +X (opts, pf, i); +X (opts, hooknum, i); +X (opts, priority, i); +X (opts, flags, i); +#undef X + link =bpf_program__attach_netfilter (prog, &opts); +if (nf_hook_attach_tests[i].expect_success) { +struct bpf_link *link2; + +if (!ASSERT_OK_PTR (link,"program attach successful" )) +continue ; + + link2 =bpf_program__attach_netfilter (prog, &opts); +ASSERT_ERR_PTR (link2,"attach program with same pf/hook/priority" ); + +if (!ASSERT_OK (bpf_link__destroy (link),"link destroy" )) +break ; + + link2 =bpf_program__attach_netfilter (prog, &opts); +if (!ASSERT_OK_PTR (link2,"program reattach successful" )) +continue ; +if (!ASSERT_OK (bpf_link__destroy (link2),"link destroy" )) +break ; + }else { +ASSERT_ERR_PTR (link,"program load failure" ); + } + } +out: +test_netfilter_link_attach__destroy (skel); +}
tools/testing/selftests/bpf/progs/test_netfilter_link_attach.c new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#include "vmlinux.h" +#include <bpf/bpf_helpers.h> + +#define NF_ACCEPT 1 + +SEC("netfilter") +int nf_link_attach_test (struct bpf_nf_ctx *ctx) +{ +return NF_ACCEPT ; +} + +char _license[] SEC("license") = "GPL";
Signature · 1 line
2.40.1
PATCH 3/7 netfilter: defrag: Add glue hooks for enabling/disabling defrag
In reply to Daniel XuWe want to be able to enable/disable IP packet defrag from core bpf/netfilter code. In other words, execute code from core that could possibly be built as a module.
To help avoid symbol resolution errors, use glue hooks that the modules will register callbacks with during module init.
--- include/linux/netfilter.h | 12 ++++++++++++ net/ipv4/netfilter/nf_defrag_ipv4.c | 8 ++++++++ net/ipv6/netfilter/nf_defrag_ipv6_hooks.c | 10 ++++++++++ net/netfilter/core.c | 6 ++++++ 4 files changed, 36 insertions(+)
include/linux/netfilter.h +12 -0
@@ -481,6 +481,18 @@ struct nfnl_ct_hook { };extern const struct nfnl_ct_hook __rcu *nfnl_ct_hook; +struct nf_defrag_v4_hook { +int (*enable)(struct net *net); +void (*disable)(struct net *net); +}; +extern const struct nf_defrag_v4_hook __rcu *nf_defrag_v4_hook; + +struct nf_defrag_v6_hook { +int (*enable)(struct net *net); +void (*disable)(struct net *net); +}; +extern const struct nf_defrag_v6_hook __rcu *nf_defrag_v6_hook; +/** * nf_skb_duplicated - TEE target has sent a packet *
net/ipv4/netfilter/nf_defrag_ipv4.c +8 -0
@@ -7,6 +7,7 @@#include <linux/ip.h> #include <linux/netfilter.h> #include <linux/module.h> +#include <linux/rcupdate.h> #include <linux/skbuff.h> #include <net/netns/generic.h> #include <net/route.h>
@@ -113,17 +114,24 @@ static void __net_exit defrag4_net_exit(struct net *net) } } +static struct nf_defrag_v4_hook defrag_hook = { + .enable = nf_defrag_ipv4_enable, + .disable = nf_defrag_ipv4_disable, +}; +static struct pernet_operations defrag4_net_ops = { .exit = defrag4_net_exit, };static int __init nf_defrag_init (void ) { +rcu_assign_pointer (nf_defrag_v4_hook, &defrag_hook);return register_pernet_subsys (&defrag4_net_ops); } static void __exit nf_defrag_fini(void) { +rcu_assign_pointer (nf_defrag_v4_hook,NULL );unregister_pernet_subsys (&defrag4_net_ops); }
net/ipv6/netfilter/nf_defrag_ipv6_hooks.c +10 -0
@@ -10,6 +10,7 @@#include <linux/module.h> #include <linux/skbuff.h> #include <linux/icmp.h> +#include <linux/rcupdate.h> #include <linux/sysctl.h> #include <net/ipv6_frag.h>
@@ -96,6 +97,11 @@ static void __net_exit defrag6_net_exit(struct net *net) } } +static struct nf_defrag_v6_hook defrag_hook = { + .enable = nf_defrag_ipv6_enable, + .disable = nf_defrag_ipv6_disable, +}; +static struct pernet_operations defrag6_net_ops = { .exit = defrag6_net_exit, };
@@ -114,6 +120,9 @@ static int __init nf_defrag_init(void)pr_err ("nf_defrag_ipv6: can't register pernet ops\n" ); goto cleanup_frag6; } + +rcu_assign_pointer (nf_defrag_v6_hook, &defrag_hook); +return ret; cleanup_frag6:
@@ -124,6 +133,7 @@ static int __init nf_defrag_init(void) static void __exit nf_defrag_fini(void) { +rcu_assign_pointer (nf_defrag_v6_hook,NULL );unregister_pernet_subsys (&defrag6_net_ops);nf_ct_frag6_cleanup (); }
net/netfilter/core.c +6 -0
@@ -680,6 +680,12 @@ EXPORT_SYMBOL_GPL(nfnl_ct_hook);const struct nf_ct_hook __rcu *nf_ct_hook__read_mostly ;EXPORT_SYMBOL_GPL (nf_ct_hook); +const struct nf_defrag_v4_hook __rcu *nf_defrag_v4_hook__read_mostly ; +EXPORT_SYMBOL_GPL (nf_defrag_v4_hook); + +const struct nf_defrag_v6_hook __rcu *nf_defrag_v6_hook__read_mostly ; +EXPORT_SYMBOL_GPL (nf_defrag_v6_hook); +#if IS_ENABLED (CONFIG_NF_CONNTRACK )u8 nf_ctnetlink_has_listener;EXPORT_SYMBOL_GPL (nf_ctnetlink_has_listener);
Signature · 1 line
2.40.1
Florian Westphal
maintainerRe: [PATCH bpf-next 3/7] netfilter: defrag: Add glue hooks for enabling/disabling defrag
Quoted 13 lines · Daniel Xu
net/ipv4/netfilter/nf_defrag_ipv4.c +1
-0@@ -7,6 +7,7 @@#include <linux/ip.h> #include <linux/netfilter.h> #include <linux/module.h> +#include <linux/rcupdate.h> #include <linux/skbuff.h> #include <net/netns/generic.h> #include <net/route.h>
net/ipv4/netfilter/nf_defrag_ipv4.c static void __net_exit defrag4_net_exit(struct net *net) · line 120
Quoted 7 lines
net/ipv4/netfilter/nf_defrag_ipv4.c +4
-0@@ -113,17 +114,24 @@ static void __net_exit defrag4_net_exit(struct net *net) } } +static struct nf_defrag_v4_hook defrag_hook = { + .enable = nf_defrag_ipv4_enable, + .disable = nf_defrag_ipv4_disable, +};
Nit: static const, same for v6.
net/ipv4/netfilter/nf_defrag_ipv4.c static void __net_exit defrag4_net_exit(struct net *net) · line 129
Quoted 8 lines
net/ipv4/netfilter/nf_defrag_ipv4.c +1
-0static struct pernet_operations defrag4_net_ops = { .exit = defrag4_net_exit, };static int __init nf_defrag_init (void ) { +rcu_assign_pointer (nf_defrag_v4_hook, &defrag_hook);return register_pernet_subsys (&defrag4_net_ops);
register_pernet failure results in nf_defrag_v4_hook pointing to garbage.
PATCH 4/7 netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link
In reply to Daniel XuThis commit adds support for enabling IP defrag using pre-existing netfilter defrag support. Basically all the flag does is bump a refcnt while the link the active. Checks are also added to ensure the prog requesting defrag support is run _after_ netfilter defrag hooks.
--- include/uapi/linux/bpf.h | 5 ++ net/netfilter/nf_bpf_link.c | 108 +++++++++++++++++++++++++++++---- tools/include/uapi/linux/bpf.h | 5 ++ 3 files changed, 107 insertions(+), 11 deletions(-)
include/uapi/linux/bpf.h +5 -0
@@ -1170,6 +1170,11 @@ enum bpf_link_type {*/ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.netfilter.flags used in LINK_CREATE command for +* BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation. +*/ +#define BPF_F_NETFILTER_IP_DEFRAG (1U << 0) +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
net/netfilter/nf_bpf_link.c +97 -11
148 lines not on this page · raw message
tools/include/uapi/linux/bpf.h +5 -0
@@ -1170,6 +1170,11 @@ enum bpf_link_type {*/ #define BPF_F_KPROBE_MULTI_RETURN (1U << 0) +/* link_create.netfilter.flags used in LINK_CREATE command for +* BPF_PROG_TYPE_NETFILTER to enable IP packet defragmentation. +*/ +#define BPF_F_NETFILTER_IP_DEFRAG (1U << 0) +/* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
Signature · 1 line
2.40.1
Florian Westphal
maintainerRe: [PATCH bpf-next 4/7] netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link
Quoted 7 lines · Daniel Xu
net/netfilter/nf_bpf_link.c +7
-0+static int bpf_nf_enable_defrag (struct bpf_nf_link *link) +{ +int err; + +switch (link->hook_ops.pf) { +#if IS_ENABLED (CONFIG_NF_DEFRAG_IPV4 ) +case NFPROTO_IPV4 :
net/netfilter/nf_bpf_link.c struct bpf_nf_link { · line 46
Quoted 10 lines
net/netfilter/nf_bpf_link.c +10
-0+const struct nf_defrag_v4_hook *v4_hook; + + err =request_module ("nf_defrag_ipv4" ); +if (err) +return err; + +rcu_read_lock (); + v4_hook =rcu_dereference (nf_defrag_v4_hook); + err = v4_hook->enable (link->net); +rcu_read_unlock ();
I'd reverse this, first try rcu_dereference(), then modprobe if thats returned NULL.
net/netfilter/nf_bpf_link.c struct bpf_nf_link { · line 80
Quoted 11 lines
net/netfilter/nf_bpf_link.c +11
-0+static void bpf_nf_disable_defrag (struct bpf_nf_link *link) +{ +switch (link->hook_ops.pf) { +#if IS_ENABLED (CONFIG_NF_DEFRAG_IPV4 ) +case NFPROTO_IPV4 : +const struct nf_defrag_v4_hook *v4_hook; + +rcu_read_lock (); + v4_hook =rcu_dereference (nf_defrag_v4_hook); + v4_hook->disable (link->net); +rcu_read_unlock ();
if (v4_hook)
v4_hook->disable()
Else we get trouble on manual 'rmmod'.
net/netfilter/nf_bpf_link.c static int bpf_nf_check_pf_and_hooks(const union bpf_attr *attr) · line 191
Quoted 9 lines
net/netfilter/nf_bpf_link.c +9
-0+/* make sure conntrack confirm is always last */ + prio = attr->link_create.netfilter.priority; +if (prio ==NF_IP_PRI_FIRST ) +return -ERANGE ;/* sabotage_in and other warts */ +else if (prio ==NF_IP_PRI_LAST ) +return -ERANGE ;/* e.g. conntrack confirm */ +else if ((attr->link_create.netfilter.flags &BPF_F_NETFILTER_IP_DEFRAG ) && + (prio >NF_IP_PRI_FIRST && prio <=NF_IP_PRI_CONNTRACK_DEFRAG )) +return -ERANGE ;/* cannot use defrag if prog runs before nf_defrag */
You could elide the (prio > NF_IP_PRI_FIRST, its already handled by first conditional. Otherwise this looks good to me.
Re: [PATCH bpf-next 4/7] netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link
Quoted 13 lines · Florian Westphal
Quoted 12 lines · Daniel Xu
net/netfilter/nf_bpf_link.c +12
-0+static int bpf_nf_enable_defrag (struct bpf_nf_link *link) +{ +int err; + +switch (link->hook_ops.pf) { +#if IS_ENABLED (CONFIG_NF_DEFRAG_IPV4 ) +case NFPROTO_IPV4 : +const struct nf_defrag_v4_hook *v4_hook; + + err =request_module ("nf_defrag_ipv4" ); +if (err) +return err;
Quoted 8 lines
net/netfilter/nf_bpf_link.c struct bpf_nf_link { · line 46
Quoted 5 lines
net/netfilter/nf_bpf_link.c +5
-0+ +rcu_read_lock (); + v4_hook =rcu_dereference (nf_defrag_v4_hook); + err = v4_hook->enable (link->net); +rcu_read_unlock ();I'd reverse this, first try
rcu_dereference(), then modprobe if thats returned NULL.
Ack.
Quoted 14 lines
net/netfilter/nf_bpf_link.c struct bpf_nf_link { · line 80
Quoted 12 lines
net/netfilter/nf_bpf_link.c +11
-0+static void bpf_nf_disable_defrag (struct bpf_nf_link *link) +{ +switch (link->hook_ops.pf) { +#if IS_ENABLED (CONFIG_NF_DEFRAG_IPV4 ) +case NFPROTO_IPV4 : +const struct nf_defrag_v4_hook *v4_hook; + +rcu_read_lock (); + v4_hook =rcu_dereference (nf_defrag_v4_hook); + v4_hook->disable (link->net); +rcu_read_unlock ();if (
v4_hook)
Quoted 3 lines
v4_hook->disable()Else we get trouble on manual 'rmmod'.
Ah good catch, thanks.
Quoted 14 lines
net/netfilter/nf_bpf_link.c static int bpf_nf_check_pf_and_hooks(const union bpf_attr *attr) · line 191
Quoted 10 lines
net/netfilter/nf_bpf_link.c +9
-0+/* make sure conntrack confirm is always last */ + prio = attr->link_create.netfilter.priority; +if (prio ==NF_IP_PRI_FIRST ) +return -ERANGE ;/* sabotage_in and other warts */ +else if (prio ==NF_IP_PRI_LAST ) +return -ERANGE ;/* e.g. conntrack confirm */ +else if ((attr->link_create.netfilter.flags &BPF_F_NETFILTER_IP_DEFRAG ) && + (prio >NF_IP_PRI_FIRST && prio <=NF_IP_PRI_CONNTRACK_DEFRAG )) +return -ERANGE ;/* cannot use defrag if prog runs before nf_defrag */ You could elide the (prio >
NF_IP_PRI_FIRST, its already handled by first conditional. Otherwise this looks good to me.
Ah, right. It's INT_MIN.
Thanks,
Daniel
PATCH 5/7 bpf: selftests: Support not connecting client socket
In reply to Daniel XuFor connectionless protocols or raw sockets we do not want to actually connect() to the server.
--- tools/testing/selftests/bpf/network_helpers.c | 5 +++-- tools/testing/selftests/bpf/network_helpers.h | 1 + 2 files changed, 4 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/network_helpers.c +3 -2
@@ -301,8 +301,9 @@ int connect_to_fd_opts(int server_fd, const struct network_helper_opts *opts) strlen(opts->cc) + 1)) goto error_close; -if (connect_fd_to_addr (fd, &addr, addrlen, opts->must_fail)) - goto error_close; +if (!opts->noconnect) +if (connect_fd_to_addr (fd, &addr, addrlen, opts->must_fail)) + goto error_close;return fd;
tools/testing/selftests/bpf/network_helpers.h +1 -0
@@ -21,6 +21,7 @@ struct network_helper_opts {const char *cc;int timeout_ms;bool must_fail; +bool noconnect; };/* ipv4 test vector */
Signature · 1 line
2.40.1
PATCH 6/7 bpf: selftests: Support custom type and proto for client sockets
In reply to Daniel XuExtend connect_to_fd_opts() to take optional type and protocol parameters for the client socket. These parameters are useful when opening a raw socket to send IP fragments.
--- tools/testing/selftests/bpf/network_helpers.c | 21 +++++++++++++------ tools/testing/selftests/bpf/network_helpers.h | 2 ++ 2 files changed, 17 insertions(+), 6 deletions(-)
tools/testing/selftests/bpf/network_helpers.c +15 -6
@@ -270,14 +270,23 @@ int connect_to_fd_opts(int server_fd, const struct network_helper_opts *opts) opts = &default_opts; optlen =sizeof (type); -if (getsockopt (server_fd,SOL_SOCKET ,SO_TYPE , &type, &optlen)) { -log_err ("getsockopt(SOL_TYPE)" ); -return -1 ; + +if (opts->type) { + type = opts->type; + }else { +if (getsockopt (server_fd,SOL_SOCKET ,SO_TYPE , &type, &optlen)) { +log_err ("getsockopt(SOL_TYPE)" ); +return -1 ; + } } -if (getsockopt (server_fd,SOL_SOCKET ,SO_PROTOCOL , &protocol, &optlen)) { -log_err ("getsockopt(SOL_PROTOCOL)" ); -return -1 ; +if (opts->proto) { + protocol = opts->proto; + }else { +if (getsockopt (server_fd,SOL_SOCKET ,SO_PROTOCOL , &protocol, &optlen)) { +log_err ("getsockopt(SOL_PROTOCOL)" ); +return -1 ; + } } addrlen =sizeof (addr);
tools/testing/selftests/bpf/network_helpers.h +2 -0
@@ -22,6 +22,8 @@ struct network_helper_opts {int timeout_ms;bool must_fail;bool noconnect; +int type; +int proto; };/* ipv4 test vector */
Signature · 1 line
2.40.1
PATCH 7/7 bpf: selftests: Add defrag selftests
In reply to Daniel XuThese selftests tests 2 major scenarios: the BPF based defragmentation can successfully be done and that packet pointers are invalidated after calls to the kfunc. The logic is similar for both ipv4 and ipv6.
In the first scenario, we create a UDP client and UDP echo server. The the server side is fairly straightforward: we attach the prog and simply echo back the message.
The on the client side, we send fragmented packets to and expect the reassembled message back from the server.
--- tools/testing/selftests/bpf/Makefile | 4 +- .../selftests/bpf/generate_udp_fragments.py | 90 ++++++ .../selftests/bpf/ip_check_defrag_frags.h | 57 ++++ .../bpf/prog_tests/ip_check_defrag.c | 282 ++++++++++++++++++ .../selftests/bpf/progs/ip_check_defrag.c | 104 +++++++ 5 files changed, 535 insertions(+), 2 deletions(-) create mode 100755 tools/testing/selftests/bpf/generate_udp_fragments.py create mode 100644 tools/testing/selftests/bpf/ip_check_defrag_frags.h create mode 100644 tools/testing/selftests/bpf/prog_tests/ip_check_defrag.c create mode 100644 tools/testing/selftests/bpf/progs/ip_check_defrag.c
tools/testing/selftests/bpf/Makefile +2 -2
@@ -561,8 +561,8 @@ TRUNNER_EXTRA_SOURCES := test_progs.c cgroup_helpers.c trace_helpers.c \ network_helpers.c testing_helpers.c \ btf_helpers.c flow_dissector_load.h \ cap_helpers.c test_loader.c xsk.c disasm.c \ - json_writer.c unpriv_helpers.c - + json_writer.c unpriv_helpers.c \ + ip_check_defrag_frags.h TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ $(OUTPUT)/liburandom_read.so \ $(OUTPUT)/xdp_synproxy \
tools/testing/selftests/bpf/generate_udp_fragments.py new file +90 -0
90 lines not on this page · raw message
tools/testing/selftests/bpf/ip_check_defrag_frags.h new file +57 -0
57 lines not on this page · raw message
tools/testing/selftests/bpf/prog_tests/ip_check_defrag.c new file +282 -0
282 lines not on this page · raw message
tools/testing/selftests/bpf/progs/ip_check_defrag.c new file +104 -0
104 lines not on this page · raw message
Signature · 1 line
2.40.1
Florian Westphal
maintainerQuoted 4 lines · Daniel Xu
Patches 1 & 2 are stolenfrom Florian. Hopefully he doesn't mind. There were some outstanding comments on the v2 [2] but it doesn't look like a v3 was ever submitted. I've addressed the comments and put them in this patchset cuz I needed them.
I did not submit a v3 because i had to wait for the bpf -> bpf-next merge to get "bpf: netfilter: Add BPF_NETFILTER bpf_attach_type".
Now that has been done so I will do v3 shortly.
Hi Florian,
Quoted 10 lines · Florian Westphal
Quoted 4 lines · Daniel Xu
Patches 1 & 2 are stolenfrom Florian. Hopefully he doesn't mind. There were some outstanding comments on the v2 [2] but it doesn't look like a v3 was ever submitted. I've addressed the comments and put them in this patchset cuz I needed them.
I did not submit a v3 because i had to wait for the bpf -> bpf-next merge to get "bpf: netfilter: Add
BPF_NETFILTERbpf_attach_type".Now that has been done so I will do v3 shortly.
Ack. Will wait for your patches to go in before sending my v2.
Thanks,
Daniel
Quoted 5 lines
The basic idea is we bump a refcnt on the netfilter defrag module and then run the bpf prog after the defrag module runs. This allows bpf progs to transparently see full, reassembled packets. The nice thing about this is that progs don't have to carry around logic to detect fragments.
One high-level comment after glancing through the series: Instead of allocating a flag specifically for the defrag module, why not support loading (and holding) arbitrary netfilter modules in the UAPI? If we need to allocate a new flag every time someone wants to use a netfilter module along with BPF we'll run out of flags pretty quickly :)
-Toke
Hi Toke,
Thanks for taking a look at the patchset.
Quoted 11 lines · Toke Høiland-Jørgensen
Quoted 5 lines
The basic idea is we bump a refcnt on the netfilter defrag module and then run the bpf prog after the defrag module runs. This allows bpf progs to transparently see full, reassembled packets. The nice thing about this is that progs don't have to carry around logic to detect fragments.
One high-level comment after glancing through the series: Instead of allocating a flag specifically for the defrag module, why not support loading (and holding) arbitrary netfilter modules in the UAPI? If we need to allocate a new flag every time someone wants to use a netfilter module along with BPF we'll run out of flags pretty quickly :)
I don't have enough context on netfilter in general to say if it'd be generically useful -- perhaps Florian can comment on that.
However, I'm not sure such a mechanism removes the need for a flag. The netfilter defrag modules still need to be called into to bump the refcnt.
The module could export some kfuncs to inc/dec the refcnt, but it'd be rather odd for prog code to think about the lifetime of the attachment (as inc/dec for _each_ prog execution seems wasteful and slow). AFAIK all the other resource acquire/release APIs are for a single prog execution.
So a flag for link attach feels the most natural to me. We could always add a flag2 field or something right?
[...]
Thanks,
Daniel
Florian Westphal
maintainerQuoted 9 lines · Toke Høiland-Jørgensen
Quoted 5 lines
The basic idea is we bump a refcnt on the netfilter defrag module and then run the bpf prog after the defrag module runs. This allows bpf progs to transparently see full, reassembled packets. The nice thing about this is that progs don't have to carry around logic to detect fragments.
One high-level comment after glancing through the series: Instead of allocating a flag specifically for the defrag module, why not support loading (and holding) arbitrary netfilter modules in the UAPI?
How would that work/look like?
defrag (and conntrack) need special handling because loading these modules has no effect on the datapath.
Traditionally, yes, loading was enough, but now with netns being ubiquitous we don't want these to get enabled unless needed.
Ignoring bpf, this happens when user adds nftables/iptables rules that check for conntrack state, use some form of NAT or use e.g. tproxy.
For bpf a flag during link attachment seemed like the best way to go.
At the moment I only see two flags for this, namely "need defrag" and "need conntrack".
For conntrack, we MIGHT be able to not need a flag but maybe verifier could "guess" based on kfuncs used.
But for defrag, I don't think its good to add a dummy do-nothing kfunc just for expressing the dependency on bpf prog side.
Quoted 16 lines · Florian Westphal
Quoted 9 lines · Toke Høiland-Jørgensen
Quoted 5 lines
The basic idea is we bump a refcnt on the netfilter defrag module and then run the bpf prog after the defrag module runs. This allows bpf progs to transparently see full, reassembled packets. The nice thing about this is that progs don't have to carry around logic to detect fragments.
One high-level comment after glancing through the series: Instead of allocating a flag specifically for the defrag module, why not support loading (and holding) arbitrary netfilter modules in the UAPI?
How would that work/look like?
defrag (and conntrack) need special handling because loading these modules has no effect on the datapath.
Quoted 8 lines
Traditionally, yes, loading was enough, but now with netns being ubiquitous we don't want these to get enabled unless needed.
Ignoring bpf, this happens when user adds nftables/iptables rules that check for conntrack state, use some form of NAT or use e.g. tproxy.
For bpf a flag during link attachment seemed like the best way to go.
Right, I wasn't disputing that having a flag to load a module was a good idea. On the contrary, I was thinking we'd need many more of these if/when BPF wants to take advantage of more netfilter code. Say, if a BPF module wants to call into TPROXY, that module would also need go be loaded and kept around, no?
I was thinking something along the lines of just having a field 'netfilter_modules[]' where userspace could put an arbitrary number of module names into, and we'd load all of them and put a ref into the bpf_link. In principle, we could just have that be a string array of module names, but that's probably a bit cumbersome (and, well, building a generic module loader interface into the bpf_like API is not desirable either). But maybe with an explicit ENUM?
Quoted 5 lines
At the moment I only see two flags for this, namely "need defrag" and "need conntrack".
For conntrack, we MIGHT be able to not need a flag but maybe verifier could "guess" based on kfuncs used.
If the verifier can just identify the modules from the kfuncs and do the whole thing automatically, that would of course be even better from an ease-of-use PoV. Not sure what that would take, though? I seem to recall having discussions around these lines before that fell down on various points.
Quoted 2 lines
But for defrag, I don't think its good to add a dummy do-nothing kfunc just for expressing the dependency on bpf prog side.
Agreed.
-Toke
Florian Westphal
maintainerQuoted 9 lines · Toke Høiland-Jørgensen
Quoted 2 lines · Florian Westphal
For bpf a flag during link attachment seemed like the best way to go.
Right, I wasn't disputing that having a flag to load a module was a good idea. On the contrary, I was thinking we'd need many more of these if/when BPF wants to take advantage of more netfilter code. Say, if a BPF module wants to call into TPROXY, that module would also need go be loaded and kept around, no?
That seems to be a different topic that has nothing to do with either bpf_link or netfilter?
If the program calls into say, TPROXY, then I'd expect that this needs to be handled via kfuncs, no? Or if I misunderstand, what do you mean by "call into TPROXY"?
And if so, thats already handled at bpf_prog load time, not at link creation time, or do I miss something here?
AFAIU, if prog uses such kfuncs, verifier will grab needed module ref and if module isn't loaded the kfuncs won't be found and program load fails.
Quoted 4 lines
I was thinking something along the lines of just having a field 'netfilter_modules[]' where userspace could put an arbitrary number of module names into, and we'd load all of them and put a ref into the
bpf_link.
Why? I fail to understand the connection between bpf_link, netfilter and modules. What makes netfilter so special that we need such a module array, and what does that have to do with bpf_link interface?
Quoted 4 lines
In principle, we could just have that be a string array f module names, but that's probably a bit cumbersome (and, well, building a generic module loader interface into the
bpf_likeAPI is not desirable either). But maybe with an explicit ENUM?
What functionality does that provide? I can't think of a single module where this functionality is needed.
Either we're talking about future kfuncs, then, as far as i understand how kfuncs work, this is handled at bpf_prog load time, not when the bpf_link is created.
Or we are talking about implicit dependencies, where program doesn't call function X but needs functionality handled earlier in the pipeline?
The only two instances I know where this is the case for netfilter is defrag + conntrack.
Quoted 8 lines
Quoted 2 lines
For conntrack, we MIGHT be able to not need a flag but maybe verifier could "guess" based on kfuncs used.
If the verifier can just identify the modules from the kfuncs and do the whole thing automatically, that would of course be even better from an ease-of-use PoV. Not sure what that would take, though? I seem to recall having discussions around these lines before that fell down on various points.
AFAICS the conntrack kfuncs are wired to nf_conntrack already, so I would expect that the module has to be loaded already for the verifier to accept the program.
Those kfuncs are not yet exposed to NETFILTER program types. Once they are, all that would be needed is for the netfilter bpf_link to be able tp detect that the prog is calling into those kfuncs, and then make the needed register/unregister calls to enable the conntrack hooks.
Wheter thats better than using an explicit "please turn on conntrack for me", I don't know. Perhaps future bpf programs could access skb->_nfct directly without kfuncs so I'd say the flag is a better approach from an uapi point of view.
Quoted 24 lines · Florian Westphal
Quoted 9 lines · Toke Høiland-Jørgensen
Quoted 2 lines · Florian Westphal
For bpf a flag during link attachment seemed like the best way to go.
Right, I wasn't disputing that having a flag to load a module was a good idea. On the contrary, I was thinking we'd need many more of these if/when BPF wants to take advantage of more netfilter code. Say, if a BPF module wants to call into TPROXY, that module would also need go be loaded and kept around, no?
That seems to be a different topic that has nothing to do with either
bpf_linkor netfilter?If the program calls into say, TPROXY, then I'd expect that this needs to be handled via kfuncs, no? Or if I misunderstand, what do you mean by "call into TPROXY"?
And if so, thats already handled at
bpf_progload time, not at link creation time, or do I miss something here?AFAIU, if prog uses such kfuncs, verifier will grab needed module ref and if module isn't loaded the kfuncs won't be found and program load fails.
...
Quoted 5 lines
Or we are talking about implicit dependencies, where program doesn't call function X but needs functionality handled earlier in the pipeline?
The only two instances I know where this is the case for netfilter is defrag + conntrack.
Well, I was kinda mixing the two cases above, sorry about that. The "kfuncs locking the module" was not present in my mind when starting to talk about that bit...
As for the original question, that's answered by your point above: If those two modules are the only ones that are likely to need this, then a flag for each is fine by me - that was the key piece I was missing (I'm not a netfilter expert, as you well know).
Thanks for clarifying, and apologies for the muddled thinking! :)
-Toke
Florian Westphal
maintainerQuoted 5 lines · Toke Høiland-Jørgensen
Florian Westphal <[email protected]> writes:
As for the original question, that's answered by your point above: If those two modules are the only ones that are likely to need this, then a flag for each is fine by me - that was the key piece I was missing (I'm not a netfilter expert, as you well know).
No problem, I was worried I was missing an important piece of kfunc plumbing :-)
You do raise a good point though. With kfuncs, module is pinned. So, should a "please turn on defrag for this bpf_link" pin the defrag modules too?
For plain netfilter we don't do that, i.e. you can just do "rmmod nf_defrag_ipv4". But I suspect that for the new bpf-link defrag we probably should grab a reference to prevent unwanted functionality breakage of the bpf prog.
Quoted 10 lines · Florian Westphal
Quoted 5 lines · Toke Høiland-Jørgensen
Florian Westphal <[email protected]> writes:
As for the original question, that's answered by your point above: If those two modules are the only ones that are likely to need this, then a flag for each is fine by me - that was the key piece I was missing (I'm not a netfilter expert, as you well know).No problem, I was worried I was missing an important piece of kfunc plumbing :-)
Quoted 8 lines
You do raise a good point though. With kfuncs, module is pinned. So, should a "please turn on defrag for this
bpf_link" pin the defrag modules too?For plain netfilter we don't do that, i.e. you can just do "rmmod
nf_defrag_ipv4". But I suspect that for the new bpf-link defrag we probably should grab a reference to prevent unwanted functionality breakage of the bpf prog.
Ack. Will add to v3.
Thanks,
Daniel