bpf: add BPF_LWT_ENCAP_IP option to bpf_lwt_push_encap
Subsystems (7)
PATCH v9 1/7 bpf: add plumbing for BPF_LWT_ENCAP_IP in bpf_lwt_push_encap
In reply to Peter OskolkovThis patch adds all needed plumbing in preparation to allowing bpf programs to do IP encapping via bpf_lwt_push_encap. Actual implementation is added in the next patch in the patchset.
Of note:
- bpf_lwt_push_encap can now be called from BPF_PROG_TYPE_LWT_XMIT prog types in addition to BPF_PROG_TYPE_LWT_IN;
- if the skb being encapped has GSO set, encapsulation is limited to IPIP/IP+GRE/IP+GUE (both IPv4 and IPv6);
- as route lookups are different for ingress vs egress, the single external bpf_lwt_push_encap BPF helper is routed internally to either bpf_lwt_in_push_encap or bpf_lwt_xmit_push_encap BPF_CALLs, depending on prog type.
v8 changes: fixed a typo.
--- include/uapi/linux/bpf.h | 26 ++++++++++++++++++++-- net/core/filter.c | 48 +++++++++++++++++++++++++++++++++++----- 2 files changed, 67 insertions(+), 7 deletions(-)
include/uapi/linux/bpf.h +24 -2
@@ -2016,6 +2016,19 @@ union bpf_attr {* Only works if *skb* contains an IPv6 packet. Insert a * Segment Routing Header (**struct ipv6_sr_hdr**) inside * the IPv6 header. +* **BPF_LWT_ENCAP_IP** +* IP encapsulation (GRE/GUE/IPIP/etc). The outer header +* must be IPv4 or IPv6, followed by zero or more +* additional headers, up to LWT_BPF_MAX_HEADROOM total +* bytes in all prepended headers. Please note that +* if skb_is_gso(skb) is true, no more than two headers +* can be prepended, and the inner header, if present, +* should be either GRE or UDP/GUE. +* +* BPF_LWT_ENCAP_SEG6*** types can be called by bpf programs of +* type BPF_PROG_TYPE_LWT_IN; BPF_LWT_ENCAP_IP type can be called +* by bpf programs of types BPF_PROG_TYPE_LWT_IN and +* BPF_PROG_TYPE_LWT_XMIT. * * A call to this helper is susceptible to change the underlaying * packet buffer. Therefore, at load time, all checks on pointers
@@ -2517,7 +2530,8 @@ enum bpf_hdr_start_off {/* Encapsulation type for BPF_FUNC_lwt_push_encap helper. */ enum bpf_lwt_encap_mode {BPF_LWT_ENCAP_SEG6 , -BPF_LWT_ENCAP_SEG6_INLINE +BPF_LWT_ENCAP_SEG6_INLINE , +BPF_LWT_ENCAP_IP , };#define __bpf_md_ptr (type, name) \
@@ -2606,7 +2620,15 @@ enum bpf_ret_code {BPF_DROP =2 ,/* 3-6 reserved */ BPF_REDIRECT =7 , - /* >127 are reserved for prog type specific return codes */ +/* >127 are reserved for prog type specific return codes. +* +* BPF_LWT_REROUTE: used by BPF_PROG_TYPE_LWT_IN and +* BPF_PROG_TYPE_LWT_XMIT to indicate that skb had been +* changed and should be routed based on its new L3 header. +* (This is an L3 redirect, as opposed to L2 redirect +* represented by BPF_REDIRECT above). +*/ + BPF_LWT_REROUTE = 128, }; struct bpf_sock {
net/core/filter.c +43 -5
84 lines not on this page · raw message
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
v9 cover bpf: add BPF_LWT_ENCAP_IP option to bpf_lwt_push_encap
This patchset implements BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap BPF helper. It enables BPF programs (specifically, BPF_PROG_TYPE_LWT_IN and BPF_PROG_TYPE_LWT_XMIT prog types) to add IP encapsulation headers to packets (e.g. IP/GRE, GUE, IPIP).
This is useful when thousands of different short-lived flows should be encapped, each with different and dynamically determined destination. Although lwtunnels can be used in some of these scenarios, the ability to dynamically generate encap headers adds more flexibility, e.g. when routing depends on the state of the host (reflected in global bpf maps).
V2 changes: added flowi-based route lookup, IPv6 encapping, and
encapping on ingress.
V3 changes: incorporated David Ahern's suggestions:
- added l3mdev check/oif (patch 2)
- sync bpf.h from include/uapi into tools/include/uapi
- selftest tweaks
V4 changes: moved route lookup/dst change from bpf_push_ip_encap
to when BPF_LWT_REROUTE is handled, as suggested by David Ahern.
V5 changes: added a check in lwt_xmit that skb->protocol stays the
same if the skb is to be passed back to the stack (ret == BPF_OK). Again, suggested by David Ahern.
V6 changes: abandoned.
V7 changes: added handling of GSO packets (patch 3 in the patchset added),
as suggested by BPF maintainers.
V8 changes:
- fixed build errors when LWT or IPV6 are not enabled;
- whitelisted TCP GSO instead of blacklisting SCTP and UDP GSO, as
suggested by Willem de Bruijn;
- added validation that pushed length cover needed headers when GRE/UDP
encap is detected, as suggested by Willem de Bruijn;- a couple of minor/stylistic tweaks/fixed typos.
V9 changes:
- fixed a kbuild test robot compiler warning;
- added ipv6_route_input to ipv6_stub (patch 4 in the patchset added), and IPv6 routing functions are now invoked via ipv6_stub, as suggested by David Ahern.
Peter Oskolkov (7):
bpf: add plumbing for BPF_LWT_ENCAP_IP in bpf_lwt_push_encap
bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap
bpf: handle GSO in bpf_lwt_push_encap
ipv6_stub: add ipv6_route_input stub/proxy.
bpf: add handling of BPF_LWT_REROUTE to lwt_bpf.c
bpf: sync <kdir>/include/.../bpf.h with tools/include/.../bpf.h
selftests: bpf: add test_lwt_ip_encap selftest
include/net/addrconf.h | 1 + include/net/lwtunnel.h | 2 + include/uapi/linux/bpf.h | 26 +- net/core/filter.c | 49 ++- net/core/lwt_bpf.c | 261 +++++++++++++++ net/ipv6/addrconf_core.c | 6 + net/ipv6/af_inet6.c | 7 + tools/include/uapi/linux/bpf.h | 26 +- tools/testing/selftests/bpf/Makefile | 5 +- .../testing/selftests/bpf/test_lwt_ip_encap.c | 85 +++++ .../selftests/bpf/test_lwt_ip_encap.sh | 311 ++++++++++++++++++ 11 files changed, 768 insertions(+), 11 deletions(-) create mode 100644 tools/testing/selftests/bpf/test_lwt_ip_encap.c create mode 100755 tools/testing/selftests/bpf/test_lwt_ip_encap.sh
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH v9 2/7 bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap
Implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap BPF helper. It enables BPF programs (specifically, BPF_PROG_TYPE_LWT_IN and BPF_PROG_TYPE_LWT_XMIT prog types) to add IP encapsulation headers to packets (e.g. IP/GRE, GUE, IPIP).
This is useful when thousands of different short-lived flows should be encapped, each with different and dynamically determined destination. Although lwtunnels can be used in some of these scenarios, the ability to dynamically generate encap headers adds more flexibility, e.g. when routing depends on the state of the host (reflected in global bpf maps).
v7 changes:
- added a call skb_clear_hash();
- removed calls to skb_set_transport_header();
- refuse to encap GSO-enabled packets.
v8 changes:
- fix build errors when LWT is not enabled.
Note: the next patch in the patchset with deal with GSO-enabled packets, which are currently rejected at encapping attempt.
--- include/net/lwtunnel.h | 2 ++ net/core/filter.c | 3 +- net/core/lwt_bpf.c | 65 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 69 insertions(+), 1 deletion(-)
include/net/lwtunnel.h +2 -0
@@ -126,6 +126,8 @@ int lwtunnel_cmp_encap(struct lwtunnel_state *a, struct lwtunnel_state *b);int lwtunnel_output (struct net *net,struct sock *sk,struct sk_buff *skb);int lwtunnel_input (struct sk_buff *skb);int lwtunnel_xmit (struct sk_buff *skb); +int bpf_lwt_push_ip_encap (struct sk_buff *skb,void *hdr,u32 len, +bool ingress);static inline void lwtunnel_set_redirect (struct dst_entry *dst) {
net/core/filter.c +2 -1
@@ -73,6 +73,7 @@#include <linux/seg6_local.h> #include <net/seg6.h> #include <net/seg6_local.h> +#include <net/lwtunnel.h> /** * sk_filter_trim_cap - run a packet through a socket filter
@@ -4819,7 +4820,7 @@ static int bpf_push_seg6_encap(struct sk_buff *skb, u32 type, void *hdr, u32 lenstatic int bpf_push_ip_encap (struct sk_buff *skb,void *hdr,u32 len,bool ingress) { -return -EINVAL ;/* Implemented in the next patch. */ +return bpf_lwt_push_ip_encap (skb, hdr, len, ingress); }#endif
net/core/lwt_bpf.c +65 -0
71 lines not on this page · raw message
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH v9 3/7 bpf: handle GSO in bpf_lwt_push_encap
In reply to Peter OskolkovThis patch adds handling of GSO packets in bpf_lwt_push_ip_encap() (called from bpf_lwt_push_encap):
* IPIP, GRE, and UDP encapsulation types are deduced by looking into iphdr->protocol or ipv6hdr->next_header;
* SCTP GSO packets are not supported (as bpf_skb_proto_4_to_6 and similar do);
* UDP_L4 GSO packets are also not supported (although they are not blocked in bpf_skb_proto_4_to_6 and similar), as skb_decrease_gso_size() will break it;
* SKB_GSO_DODGY bit is set.
Note: it may be possible to support SCTP and UDP_L4 gso packets;
but as these cases seem to be not well handled by other tunneling/encapping code paths, the solution should be generic enough to apply to all tunneling/encapping code.
v8 changes:
- make sure that if GRE or UDP encap is detected, there is
enough of pushed bytes to cover both IP[v6] + GRE|UDP headers;
- do not reject double-encapped packets;
- whitelist TCP GSO packets rather than block SCTP GSO and UDP GSO.
--- net/core/lwt_bpf.c | 67 ++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 65 insertions(+), 2 deletions(-)
net/core/lwt_bpf.c +65 -2
@@ -16,6 +16,7 @@#include <linux/types.h> #include <linux/bpf.h> #include <net/lwtunnel.h> +#include <net/gre.h> struct bpf_lwt_prog {struct bpf_prog *prog;
@@ -390,10 +391,72 @@ static const struct lwtunnel_encap_ops bpf_encap_ops = { .owner = THIS_MODULE, }; +static int handle_gso_type (struct sk_buff *skb,unsigned int gso_type, +int encap_len) +{ +struct skb_shared_info *shinfo =skb_shinfo (skb); + + gso_type |=SKB_GSO_DODGY ; + shinfo->gso_type |= gso_type; +skb_decrease_gso_size (shinfo, encap_len); + shinfo->gso_segs =0 ; +return 0 ; +} +static int handle_gso_encap (struct sk_buff *skb,bool ipv4,int encap_len) { -/* Handling of GSO-enabled packets is added in the next patch. */ -return -EOPNOTSUPP ; +int next_hdr_offset; +void *next_hdr; +__u8 protocol; + +/* SCTP and UDP_L4 gso need more nuanced handling than what +* handle_gso_type() does above: skb_decrease_gso_size() is not enough. +* So at the moment only TCP GSO packets are let through. +*/ +if (!(skb_shinfo (skb)->gso_type & (SKB_GSO_TCPV4 |SKB_GSO_TCPV6 ))) +return -ENOTSUPP ; + +if (ipv4) { + protocol =ip_hdr (skb)->protocol; + next_hdr_offset =sizeof (struct iphdr ); + next_hdr =skb_network_header (skb) + next_hdr_offset; + }else { + protocol =ipv6_hdr (skb)->nexthdr; + next_hdr_offset =sizeof (struct ipv6hdr ); + next_hdr =skb_network_header (skb) + next_hdr_offset; + } + +switch (protocol) { +case IPPROTO_GRE : + next_hdr_offset +=sizeof (struct gre_base_hdr ); +if (next_hdr_offset > encap_len) +return -EINVAL ; + +if (((struct gre_base_hdr *)next_hdr)->flags &GRE_CSUM ) +return handle_gso_type (skb,SKB_GSO_GRE_CSUM , + encap_len); +return handle_gso_type (skb,SKB_GSO_GRE , encap_len); + +case IPPROTO_UDP : + next_hdr_offset +=sizeof (struct udphdr ); +if (next_hdr_offset > encap_len) +return -EINVAL ; + +if (((struct udphdr *)next_hdr)->check) +return handle_gso_type (skb,SKB_GSO_UDP_TUNNEL_CSUM , + encap_len); +return handle_gso_type (skb,SKB_GSO_UDP_TUNNEL , encap_len); + +case IPPROTO_IP : +case IPPROTO_IPV6 : +if (ipv4) +return handle_gso_type (skb,SKB_GSO_IPXIP4 , encap_len); +else +return handle_gso_type (skb,SKB_GSO_IPXIP6 , encap_len); + +default : +return -EPROTONOSUPPORT ; + } }int bpf_lwt_push_ip_encap (struct sk_buff *skb,void *hdr,u32 len,bool ingress)
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH v9 4/7 ipv6_stub: add ipv6_route_input stub/proxy.
In reply to Peter OskolkovProxy ip6_route_input via ipv6_stub, for later use by lwt bpf ip encap (see the next patch in the patchset).
--- include/net/addrconf.h | 1 + net/ipv6/addrconf_core.c | 6 ++++++ net/ipv6/af_inet6.c | 7 +++++++ 3 files changed, 14 insertions(+)
include/net/addrconf.h +1 -0
@@ -248,6 +248,7 @@ struct ipv6_stub { const struct in6_addr *addr);int (*ipv6_dst_lookup)(struct net *net,struct sock *sk,struct dst_entry **dst,struct flowi6 *fl6); +int (*ipv6_route_input)(struct sk_buff *skb);struct fib6_table *(*fib6_get_table)(struct net *net,u32 id); struct fib6_info *(*fib6_lookup)(struct net *net, int oif,
net/ipv6/addrconf_core.c +6 -0
@@ -134,6 +134,11 @@ static int eafnosupport_ipv6_dst_lookup(struct net *net, struct sock *u1,return -EAFNOSUPPORT ; } +static int eafnosupport_ipv6_route_input (struct sk_buff *skb) +{ +return -EAFNOSUPPORT ; +} +static struct fib6_table *eafnosupport_fib6_get_table (struct net *net,u32 id) {return NULL ;
@@ -170,6 +175,7 @@ eafnosupport_ip6_mtu_from_fib6(struct fib6_info *f6i, struct in6_addr *daddr,const struct ipv6_stub *ipv6_stub__read_mostly = &(struct ipv6_stub ) { .ipv6_dst_lookup = eafnosupport_ipv6_dst_lookup, + .ipv6_route_input = eafnosupport_ipv6_route_input, .fib6_get_table = eafnosupport_fib6_get_table, .fib6_table_lookup = eafnosupport_fib6_table_lookup, .fib6_lookup = eafnosupport_fib6_lookup,
net/ipv6/af_inet6.c +7 -0
@@ -900,10 +900,17 @@ static struct pernet_operations inet6_net_ops = { .exit = inet6_net_exit, }; +static int ipv6_route_input (struct sk_buff *skb) +{ +ip6_route_input (skb); +return skb_dst (skb)->error; +} +static const struct ipv6_stub ipv6_stub_impl = { .ipv6_sock_mc_join = ipv6_sock_mc_join, .ipv6_sock_mc_drop = ipv6_sock_mc_drop, .ipv6_dst_lookup = ip6_dst_lookup, + .ipv6_route_input = ipv6_route_input, .fib6_get_table = fib6_get_table, .fib6_table_lookup = fib6_table_lookup, .fib6_lookup = fib6_lookup,
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH v9 5/7 bpf: add handling of BPF_LWT_REROUTE to lwt_bpf.c
In reply to Peter OskolkovThis patch builds on top of the previous patch in the patchset, which added BPF_LWT_ENCAP_IP mode to bpf_lwt_push_encap. As the encapping can result in the skb needing to go via a different interface/route/dst, bpf programs can indicate this by returning BPF_LWT_REROUTE, which triggers a new route lookup for the skb.
v8 changes: fix kbuild errors when LWTUNNEL_BPF is builtin, but
IPV6 is a module: as LWTUNNEL_BPF can only be either Y or N, call IPV6 routing functions only if they are built-in.
v9 changes:
- fixed a kbuild test robot compiler warning;
- call IPV6 routing functions via ipv6_stub.
--- net/core/lwt_bpf.c | 133 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 133 insertions(+)
net/core/lwt_bpf.c +133 -0
@@ -17,6 +17,7 @@#include <linux/bpf.h> #include <net/lwtunnel.h> #include <net/gre.h> +#include <net/ip6_route.h> struct bpf_lwt_prog {struct bpf_prog *prog;
@@ -56,6 +57,7 @@ static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt,switch (ret) {case BPF_OK : +case BPF_LWT_REROUTE :break ;case BPF_REDIRECT :
@@ -88,6 +90,35 @@ static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt,return ret; } +static int bpf_lwt_input_reroute (struct sk_buff *skb) +{ +int err = -EINVAL ; + +if (skb->protocol ==htons (ETH_P_IP )) { +struct iphdr *iph =ip_hdr (skb); + + err =ip_route_input_noref (skb, iph->daddr, iph->saddr, + iph->tos,skb_dst (skb)->dev); + }else if (skb->protocol ==htons (ETH_P_IPV6 )) { +#if IS_ENABLED (CONFIG_IPV6 ) + err = ipv6_stub->ipv6_route_input (skb); +#else +pr_warn_once ("BPF_LWT_REROUTE input: IPV6 not available\n" ); +#endif + }else { +pr_warn_once ("BPF_LWT_REROUTE input: unsupported proto %d\n" , + skb->protocol); + } + +if (err) + goto err; +return dst_input (skb); + +err: +kfree_skb (skb); +return err; +} +static int bpf_input (struct sk_buff *skb) {struct dst_entry *dst =skb_dst (skb);
@@ -99,6 +130,8 @@ static int bpf_input(struct sk_buff *skb) ret =run_lwt_bpf (skb, &bpf->in, dst,NO_REDIRECT );if (ret <0 )return ret; +if (ret ==BPF_LWT_REROUTE ) +return bpf_lwt_input_reroute (skb); }if (unlikely (!dst->lwtstate->orig_input)) {
@@ -148,6 +181,95 @@ static int xmit_check_hhlen(struct sk_buff *skb)return 0 ; } +static int bpf_lwt_xmit_reroute (struct sk_buff *skb) +{ +struct net_device *l3mdev =l3mdev_master_dev_rcu (skb_dst (skb)->dev); +int oif = l3mdev ? l3mdev->ifindex :0 ; +struct dst_entry *dst =NULL ; +struct sock *sk; +struct net *net; +bool ipv4; +int err; + +if (skb->protocol ==htons (ETH_P_IP )) { + ipv4 = true; + }else if (skb->protocol ==htons (ETH_P_IPV6 )) { + ipv4 = false; + }else { +pr_warn_once ("BPF_LWT_REROUTE xmit: unsupported proto %d\n" , + skb->protocol); +return -EINVAL ; + } + + sk =sk_to_full_sk (skb->sk); +if (sk) { +if (sk->sk_bound_dev_if) + oif = sk->sk_bound_dev_if; + net =sock_net (sk); + }else { + net =dev_net (skb_dst (skb)->dev); + } + +if (ipv4) { +struct iphdr *iph =ip_hdr (skb); +struct flowi4 fl4 = {}; +struct rtable *rt; + + fl4.flowi4_oif = oif; + fl4.flowi4_mark = skb->mark; + fl4.flowi4_uid =sock_net_uid (net, sk); + fl4.flowi4_tos =RT_TOS (iph->tos); + fl4.flowi4_flags =FLOWI_FLAG_ANYSRC ; + fl4.flowi4_proto = iph->protocol; + fl4.daddr = iph->daddr; + fl4.saddr = iph->saddr; + + rt =ip_route_output_key (net, &fl4); +if (IS_ERR (rt) || rt->dst.error) +return -EINVAL ; + dst = &rt->dst; + }else { +#if IS_ENABLED (CONFIG_IPV6 ) +struct ipv6hdr *iph6 =ipv6_hdr (skb); +struct flowi6 fl6 = {}; + + fl6.flowi6_oif = oif; + fl6.flowi6_mark = skb->mark; + fl6.flowi6_uid =sock_net_uid (net, sk); + fl6.flowlabel =ip6_flowinfo (iph6); + fl6.flowi6_proto = iph6->nexthdr; + fl6.daddr = iph6->daddr; + fl6.saddr = iph6->saddr; + + err = ipv6_stub->ipv6_dst_lookup (net, skb->sk, &dst, &fl6); +if (err ||IS_ERR (dst) || dst->error) +return -EINVAL ; +#else +pr_warn_once ("BPF_LWT_REROUTE xmit: IPV6 not available\n" ); +return -EINVAL ; +#endif + } + +/* Although skb header was reserved in bpf_lwt_push_ip_encap(), it +* was done for the previous dst, so we are doing it here again, in +* case the new dst needs much more space. The call below is a noop +* if there is enough header space in skb. +*/ + err =skb_cow_head (skb,LL_RESERVED_SPACE (dst->dev)); +if (unlikely (err)) +return err; + +skb_dst_drop (skb); +skb_dst_set (skb, dst); + + err =dst_output (dev_net (skb_dst (skb)->dev), skb->sk, skb); +if (unlikely (err)) +return err; + +/* ip[6]_finish_output2 understand LWTUNNEL_XMIT_DONE */ +return LWTUNNEL_XMIT_DONE ; +} +static int bpf_xmit (struct sk_buff *skb) {struct dst_entry *dst =skb_dst (skb);
@@ -155,11 +277,20 @@ static int bpf_xmit(struct sk_buff *skb) bpf =bpf_lwt_lwtunnel (dst->lwtstate);if (bpf->xmit.prog) { +__be16 proto = skb->protocol;int ret; ret =run_lwt_bpf (skb, &bpf->xmit, dst,CAN_REDIRECT );switch (ret) {case BPF_OK : +/* If the header changed, e.g. via bpf_lwt_push_encap, +* BPF_LWT_REROUTE below should have been used if the +* protocol was also changed. +*/ +if (skb->protocol != proto) { +kfree_skb (skb); +return -EINVAL ; + }/* If the header was expanded, headroom might be too * small for L2 header to come, expand as needed. */
@@ -170,6 +301,8 @@ static int bpf_xmit(struct sk_buff *skb)return LWTUNNEL_XMIT_CONTINUE ;case BPF_REDIRECT :return LWTUNNEL_XMIT_DONE ; +case BPF_LWT_REROUTE : +return bpf_lwt_xmit_reroute (skb);default :return ret; }
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH v9 6/7 bpf: sync <kdir>/include/.../bpf.h with tools/include/.../bpf.h
In reply to Peter OskolkovThis patch copies changes in bpf.h done by a previous patch in this patchset from the kernel uapi include dir into tools uapi include dir.
--- tools/include/uapi/linux/bpf.h | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-)
tools/include/uapi/linux/bpf.h +24 -2
@@ -2016,6 +2016,19 @@ union bpf_attr {* Only works if *skb* contains an IPv6 packet. Insert a * Segment Routing Header (**struct ipv6_sr_hdr**) inside * the IPv6 header. +* **BPF_LWT_ENCAP_IP** +* IP encapsulation (GRE/GUE/IPIP/etc). The outer header +* must be IPv4 or IPv6, followed by zero or more +* additional headers, up to LWT_BPF_MAX_HEADROOM total +* bytes in all prepended headers. Please note that +* if skb_is_gso(skb) is true, no more than two headers +* can be prepended, and the inner header, if present, +* should be either GRE or UDP/GUE. +* +* BPF_LWT_ENCAP_SEG6*** types can be called by bpf programs of +* type BPF_PROG_TYPE_LWT_IN; BPF_LWT_ENCAP_IP type can be called +* by bpf programs of types BPF_PROG_TYPE_LWT_IN and +* BPF_PROG_TYPE_LWT_XMIT. * * A call to this helper is susceptible to change the underlaying * packet buffer. Therefore, at load time, all checks on pointers
@@ -2517,7 +2530,8 @@ enum bpf_hdr_start_off {/* Encapsulation type for BPF_FUNC_lwt_push_encap helper. */ enum bpf_lwt_encap_mode {BPF_LWT_ENCAP_SEG6 , -BPF_LWT_ENCAP_SEG6_INLINE +BPF_LWT_ENCAP_SEG6_INLINE , +BPF_LWT_ENCAP_IP , };#define __bpf_md_ptr (type, name) \
@@ -2606,7 +2620,15 @@ enum bpf_ret_code {BPF_DROP =2 ,/* 3-6 reserved */ BPF_REDIRECT =7 , - /* >127 are reserved for prog type specific return codes */ +/* >127 are reserved for prog type specific return codes. +* +* BPF_LWT_REROUTE: used by BPF_PROG_TYPE_LWT_IN and +* BPF_PROG_TYPE_LWT_XMIT to indicate that skb had been +* changed and should be routed based on its new L3 header. +* (This is an L3 redirect, as opposed to L2 redirect +* represented by BPF_REDIRECT above). +*/ + BPF_LWT_REROUTE = 128, }; struct bpf_sock {
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
PATCH v9 7/7 selftests: bpf: add test_lwt_ip_encap selftest
In reply to Peter OskolkovThis patch adds a bpf self-test to cover BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap.
Covered:
- encapping in LWT_IN and LWT_XMIT
- IPv4 and IPv6
A follow-up patch will add GSO and VRF-enabled tests.
--- tools/testing/selftests/bpf/Makefile | 5 +- .../testing/selftests/bpf/test_lwt_ip_encap.c | 85 +++++ .../selftests/bpf/test_lwt_ip_encap.sh | 311 ++++++++++++++++++ 3 files changed, 399 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/test_lwt_ip_encap.c create mode 100755 tools/testing/selftests/bpf/test_lwt_ip_encap.sh
tools/testing/selftests/bpf/Makefile +3 -2
@@ -36,7 +36,7 @@ BPF_OBJ_FILES = \ get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \ test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o test_xdp_vlan.o \ xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o \ - test_sock_fields_kern.o + test_sock_fields_kern.o test_lwt_ip_encap.o # Objects are built with default compilation flags and with sub-register # code-gen enabled.
@@ -74,7 +74,8 @@ TEST_PROGS := test_kmod.sh \ test_lirc_mode2.sh \ test_skb_cgroup_id.sh \ test_flow_dissector.sh \ - test_xdp_vlan.sh + test_xdp_vlan.sh \ + test_lwt_ip_encap.sh TEST_PROGS_EXTENDED := with_addr.sh \ with_tunnels.sh \
tools/testing/selftests/bpf/test_lwt_ip_encap.c new file +85 -0
85 lines not on this page · raw message
tools/testing/selftests/bpf/test_lwt_ip_encap.sh new file +311 -0
311 lines not on this page · raw message
Signature · 1 line
2.20.1.791.gb4d0f1c61a-goog
Re: [PATCH bpf-next v9 7/7] selftests: bpf: add test_lwt_ip_encap selftest
Quoted 23 lines · Peter Oskolkov
This patch adds a bpf self-test to cover
BPF_LWT_ENCAP_IPmode inbpf_lwt_push_encap.Covered:
- encapping inLWT_INandLWT_XMIT
- IPv4 and IPv6A follow-up patch will add GSO and VRF-enabled tests.
Signed-off-by Peter Oskolkov <[email protected]>--- tools/testing/selftests/bpf/Makefile | 5 +- .../testing/selftests/bpf/test_lwt_ip_encap.c | 85 +++++ .../selftests/bpf/test_lwt_ip_encap.sh | 311 ++++++++++++++++++ 3 files changed, 399 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/bpf/test_lwt_ip_encap.c create mode 100755 tools/testing/selftests/bpf/test_lwt_ip_encap.shtools/testing/selftests/bpf/Makefile +0
-0
tools/testing/selftests/bpf/Makefile BPF_OBJ_FILES = \ · line 39
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +1
-1@@ -36,7 +36,7 @@ BPF_OBJ_FILES = \ get_cgroup_id_kern.o socket_cookie_prog.o test_select_reuseport_kern.o \ test_skb_cgroup_id_kern.o bpf_flow.o netcnt_prog.o test_xdp_vlan.o \ xdp_dummy.o test_map_in_map.o test_spin_lock.o test_map_lock.o \ - test_sock_fields_kern.o + test_sock_fields_kern.o test_lwt_ip_encap.o
This needs one more rebase on top of Jiong's makefile changes. Other than this it looks great.
Thanks!
Re: [PATCH bpf-next v9 5/7] bpf: add handling of BPF_LWT_REROUTE to lwt_bpf.c
In reply to Peter OskolkovQuoted 9 lines · Peter Oskolkov
net/core/lwt_bpf.c +5
-0@@ -88,6 +90,35 @@ static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt,return ret; } +static int bpf_lwt_input_reroute (struct sk_buff *skb) +{ +int err = -EINVAL ; + +if (skb->protocol ==htons (ETH_P_IP )) {
net/core/lwt_bpf.c static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt, · line 107
Quoted 10 lines
net/core/lwt_bpf.c +10
-0+struct iphdr *iph =ip_hdr (skb); + + err =ip_route_input_noref (skb, iph->daddr, iph->saddr, + iph->tos,skb_dst (skb)->dev); + }else if (skb->protocol ==htons (ETH_P_IPV6 )) { +#if IS_ENABLED (CONFIG_IPV6 ) + err = ipv6_stub->ipv6_route_input (skb); +#else +pr_warn_once ("BPF_LWT_REROUTE input: IPV6 not available\n" ); +#endif
The stub defines ipv6_route_input when IPv6 is not enabled. addrconf_core.o is compiled in if NET is enabled irregardless of CONFIG_IPV6. ie., you don't need the IS_ENABLED check here.
If a bpf program pushes a v6 header the stub returns -EAFNOSUPPORT based on patch 4.
net/core/lwt_bpf.c static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt, · line 110
Quoted 3 lines
net/core/lwt_bpf.c +3
-0+ }else { +pr_warn_once ("BPF_LWT_REROUTE input: unsupported proto %d\n" , + skb->protocol);
You don't need a warn on here; just return -EAFNOSUPPORT.
Quoted 34 lines
net/core/lwt_bpf.c +20
-0+ } + +if (err) + goto err; +return dst_input (skb); + +err: +kfree_skb (skb); +return err; +} +static int bpf_input (struct sk_buff *skb) {struct dst_entry *dst =skb_dst (skb);@@ -99,6 +130,8 @@ static int bpf_input(struct sk_buff *skb) ret =run_lwt_bpf (skb, &bpf->in, dst,NO_REDIRECT );if (ret <0 )return ret; +if (ret ==BPF_LWT_REROUTE ) +return bpf_lwt_input_reroute (skb); }if (unlikely (!dst->lwtstate->orig_input)) {@@ -148,6 +181,95 @@ static int xmit_check_hhlen(struct sk_buff *skb)return 0 ; } +static int bpf_lwt_xmit_reroute (struct sk_buff *skb) +{ +struct net_device *l3mdev =l3mdev_master_dev_rcu (skb_dst (skb)->dev); +int oif = l3mdev ? l3mdev->ifindex :0 ; +struct dst_entry *dst =NULL ; +struct sock *sk; +struct net *net;
net/core/lwt_bpf.c static int xmit_check_hhlen(struct sk_buff *skb) · line 200
Quoted 10 lines
net/core/lwt_bpf.c +10
-0+bool ipv4; +int err; + +if (skb->protocol ==htons (ETH_P_IP )) { + ipv4 = true; + }else if (skb->protocol ==htons (ETH_P_IPV6 )) { + ipv4 = false; + }else { +pr_warn_once ("BPF_LWT_REROUTE xmit: unsupported proto %d\n" , + skb->protocol);
same here - no warn on.
Quoted 40 lines
net/core/lwt_bpf.c +40
-0+return -EINVAL ; + } + + sk =sk_to_full_sk (skb->sk); +if (sk) { +if (sk->sk_bound_dev_if) + oif = sk->sk_bound_dev_if; + net =sock_net (sk); + }else { + net =dev_net (skb_dst (skb)->dev); + } + +if (ipv4) { +struct iphdr *iph =ip_hdr (skb); +struct flowi4 fl4 = {}; +struct rtable *rt; + + fl4.flowi4_oif = oif; + fl4.flowi4_mark = skb->mark; + fl4.flowi4_uid =sock_net_uid (net, sk); + fl4.flowi4_tos =RT_TOS (iph->tos); + fl4.flowi4_flags =FLOWI_FLAG_ANYSRC ; + fl4.flowi4_proto = iph->protocol; + fl4.daddr = iph->daddr; + fl4.saddr = iph->saddr; + + rt =ip_route_output_key (net, &fl4); +if (IS_ERR (rt) || rt->dst.error) +return -EINVAL ; + dst = &rt->dst; + }else { +#if IS_ENABLED (CONFIG_IPV6 ) +struct ipv6hdr *iph6 =ipv6_hdr (skb); +struct flowi6 fl6 = {}; + + fl6.flowi6_oif = oif; + fl6.flowi6_mark = skb->mark; + fl6.flowi6_uid =sock_net_uid (net, sk); + fl6.flowlabel =ip6_flowinfo (iph6); + fl6.flowi6_proto = iph6->nexthdr;
net/core/lwt_bpf.c static int xmit_check_hhlen(struct sk_buff *skb) · line 250
Quoted 10 lines
net/core/lwt_bpf.c +10
-0+ fl6.daddr = iph6->daddr; + fl6.saddr = iph6->saddr; + + err = ipv6_stub->ipv6_dst_lookup (net, skb->sk, &dst, &fl6); +if (err ||IS_ERR (dst) || dst->error) +return -EINVAL ; +#else +pr_warn_once ("BPF_LWT_REROUTE xmit: IPV6 not available\n" ); +return -EINVAL ; +#endif
No #if .. #endif needed. The stub handles it.
Re: [PATCH bpf-next v9 7/7] selftests: bpf: add test_lwt_ip_encap selftest
In reply to Alexei StarovoitovQuoted 3 lines · Alexei Starovoitov
This needs one more rebase on top of Jiong's makefile changes. Other than this it looks great.
Thanks!
needs more than that. just sent comments about patch 5.
Re: [PATCH bpf-next v9 7/7] selftests: bpf: add test_lwt_ip_encap selftest
Quoted 7 lines · David Ahern
Quoted 3 lines · Alexei Starovoitov
This needs one more rebase on top of Jiong's makefile changes. Other than this it looks great.
Thanks!needs more than that. just sent comments about patch 5.
yep. just saw your comment. agree.