Add FOU support for externally controlled ipip devices
Subsystems (6)
cover Add FOU support for externally controlled ipip devices
This patch set adds support for using FOU or GUE encapsulation with an ipip device operating in collect-metadata mode and a set of kfuncs for controlling encap parameters exposed to a BPF tc-hook.
BPF tc-hooks allow us to read tunnel metadata (like remote IP addresses) in the ingress path of an externally controlled tunnel interface via the bpf_skb_get_tunnel_{key,opt} bpf-helpers. Packets can then be redirected to the same or a different externally controlled tunnel interface by overwriting metadata via the bpf_skb_set_tunnel_{key,opt} helpers and a call to bpf_redirect. This enables us to redirect packets between tunnel interfaces - and potentially change the encapsulation type - using only a single BPF program.
Today this approach works fine for a couple of tunnel combinations. For example: redirecting packets between Geneve and GRE interfaces or GRE and plain ipip interfaces. However, redirecting using FOU or GUE is not supported today. The ip_tunnel module does not allow us to egress packets using additional UDP encapsulation from an ipip device in collect-metadata mode.
Patch 1 lifts this restriction by adding a struct ip_tunnel_encap to the tunnel metadata. It can be filled by a new BPF kfunc introduced in Patch 2 and evaluated by the ip_tunnel egress path. This will allow us to use FOU and GUE encap with externally controlled ipip devices.
Patch 2 introduces two new BPF kfuncs: bpf_skb_{set,get}_fou_encap. These helpers can be used to set and get UDP encap parameters from the BPF tc-hook doing the packet redirect.
Patch 3 adds BPF tunnel selftests using the two kfuncs.
Christian Ehrig (3):
ipip,ip_tunnel,sit: Add FOU support for externally controlled ipip
devices
bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs selftests/bpf: Test FOU kfuncs for externally controlled ipip devices
include/net/fou.h | 2 + include/net/ip_tunnels.h | 27 ++-- net/ipv4/Makefile | 2 +- net/ipv4/fou_bpf.c | 118 ++++++++++++++++++ net/ipv4/fou_core.c | 5 + net/ipv4/ip_tunnel.c | 22 +++- net/ipv4/ipip.c | 1 + net/ipv6/sit.c | 2 +- .../selftests/bpf/progs/test_tunnel_kern.c | 117 +++++++++++++++++ tools/testing/selftests/bpf/test_tunnel.sh | 81 ++++++++++++ 10 files changed, 360 insertions(+), 17 deletions(-) create mode 100644 net/ipv4/fou_bpf.c
Signature · 1 line
2.39.2
PATCH 1/3 ipip,ip_tunnel,sit: Add FOU support for externally controlled ipip devices
Today ipip devices in collect-metadata mode don't allow for sending FOU or GUE encapsulated packets. This patch lifts the restriction by adding a struct ip_tunnel_encap to the tunnel metadata.
On the egress path, the members of this struct can be set by the bpf_skb_set_fou_encap kfunc via a BPF tc-hook. Instead of dropping packets wishing to use additional UDP encapsulation, ip_md_tunnel_xmit now evaluates the contents of this struct and adds the corresponding FOU or GUE header. Furthermore, it is making sure that additional header bytes are taken into account for PMTU discovery.
On the ingress path, an ipip device in collect-metadata mode will fill this struct and a BPF tc-hook can obtain the information via a call to the bpf_skb_get_fou_encap kfunc.
The minor change to ip_tunnel_encap, which now takes a pointer to struct ip_tunnel_encap instead of struct ip_tunnel, allows us to control FOU encap type and parameters on a per packet-level.
--- include/net/ip_tunnels.h | 27 ++++++++++++++------------- net/ipv4/ip_tunnel.c | 22 ++++++++++++++++++++-- net/ipv4/ipip.c | 1 + net/ipv6/sit.c | 2 +- 4 files changed, 36 insertions(+), 16 deletions(-)
include/net/ip_tunnels.h +14 -13
@@ -57,6 +57,13 @@ struct ip_tunnel_key {__u8 flow_flags; }; +struct ip_tunnel_encap { +u16 type; +u16 flags; +__be16 sport; +__be16 dport; +}; +/* Flags for ip_tunnel_info mode. */ #define IP_TUNNEL_INFO_TX 0x01/* represents tx tunnel parameters */ #define IP_TUNNEL_INFO_IPV6 0x02/* key contains IPv6 addresses */
@@ -66,9 +73,9 @@ struct ip_tunnel_key {#define IP_TUNNEL_OPTS_MAX \ GENMASK((sizeof_field(struct ip_tunnel_info, \ options_len) * BITS_PER_BYTE) - 1, 0) -struct ip_tunnel_info {struct ip_tunnel_key key; +struct ip_tunnel_encap encap;#ifdef CONFIG_DST_CACHE struct dst_cache dst_cache;#endif
@@ -86,13 +93,6 @@ struct ip_tunnel_6rd_parm { };#endif -struct ip_tunnel_encap { -u16 type; -u16 flags; -__be16 sport; -__be16 dport; -}; -struct ip_tunnel_prl_entry {struct ip_tunnel_prl_entry __rcu *next;__be32 addr;
@@ -293,6 +293,7 @@ struct ip_tunnel *ip_tunnel_lookup(struct ip_tunnel_net *itn, __be32 remote, __be32 local, __be32 key); +void ip_tunnel_md_udp_encap (struct sk_buff *skb,struct ip_tunnel_info *info);int ip_tunnel_rcv (struct ip_tunnel *tunnel,struct sk_buff *skb,const struct tnl_ptk_info *tpi,struct metadata_dst *tun_dst,bool log_ecn_error);
@@ -371,22 +372,22 @@ static inline int ip_encap_hlen(struct ip_tunnel_encap *e)return hlen; } -static inline int ip_tunnel_encap (struct sk_buff *skb,struct ip_tunnel *t, +static inline int ip_tunnel_encap (struct sk_buff *skb,struct ip_tunnel_encap *e,u8 *protocol,struct flowi4 *fl4) {const struct ip_tunnel_encap_ops *ops;int ret = -EINVAL ; -if (t->encap.type ==TUNNEL_ENCAP_NONE ) +if (e->type ==TUNNEL_ENCAP_NONE )return 0 ; -if (t->encap.type >=MAX_IPTUN_ENCAP_OPS ) +if (e->type >=MAX_IPTUN_ENCAP_OPS )return -EINVAL ;rcu_read_lock (); - ops =rcu_dereference (iptun_encaps[t->encap.type]); + ops =rcu_dereference (iptun_encaps[e->type]);if (likely (ops && ops->build_header)) - ret = ops->build_header (skb, &t->encap, protocol, fl4); + ret = ops->build_header (skb, e, protocol, fl4);rcu_read_unlock ();return ret;
net/ipv4/ip_tunnel.c +20 -2
@@ -359,6 +359,20 @@ static struct ip_tunnel *ip_tunnel_create(struct net *net,return ERR_PTR (err); } +void ip_tunnel_md_udp_encap (struct sk_buff *skb,struct ip_tunnel_info *info) +{ +const struct iphdr *iph =ip_hdr (skb); +const struct udphdr *udph; + +if (iph->protocol !=IPPROTO_UDP ) +return ; + + udph = (struct udphdr *)((__u8 *)iph + (iph->ihl <<2 )); + info->encap.sport = udph->source; + info->encap.dport = udph->dest; +} +EXPORT_SYMBOL (ip_tunnel_md_udp_encap); +int ip_tunnel_rcv (struct ip_tunnel *tunnel,struct sk_buff *skb,const struct tnl_ptk_info *tpi,struct metadata_dst *tun_dst,bool log_ecn_error)
@@ -572,7 +586,11 @@ void ip_md_tunnel_xmit(struct sk_buff *skb, struct net_device *dev,tunnel_id_to_key32 (key->tun_id),RT_TOS (tos),dev_net (dev),0 , skb->mark,skb_get_hash (skb), key->flow_flags); -if (tunnel->encap.type !=TUNNEL_ENCAP_NONE ) + +if (!tunnel_hlen) + tunnel_hlen =ip_encap_hlen (&tun_info->encap); + +if (ip_tunnel_encap (skb, &tun_info->encap, &proto, &fl4) <0 ) goto tx_error; use_cache =ip_tunnel_dst_cache_usable (skb, tun_info);
@@ -732,7 +750,7 @@ void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev,dev_net (dev), tunnel->parms.link, tunnel->fwmark, skb_get_hash(skb), 0); -if (ip_tunnel_encap (skb, tunnel, &protocol, &fl4) <0 ) +if (ip_tunnel_encap (skb, &tunnel->encap, &protocol, &fl4) <0 ) goto tx_error;if (connected && md) {
net/ipv4/ipip.c +1 -0
@@ -241,6 +241,7 @@ static int ipip_tunnel_rcv(struct sk_buff *skb, u8 ipproto) tun_dst =ip_tun_rx_dst (skb,0 ,0 ,0 );if (!tun_dst)return 0 ; +ip_tunnel_md_udp_encap (skb, &tun_dst->u.tun_info); }skb_reset_mac_header (skb);
net/ipv6/sit.c +1 -1
@@ -1024,7 +1024,7 @@ static netdev_tx_t ipip6_tunnel_xmit(struct sk_buff *skb, ttl = iph6->hop_limit; tos =INET_ECN_encapsulate (tos,ipv6_get_dsfield (iph6)); -if (ip_tunnel_encap (skb, tunnel, &protocol, &fl4) <0 ) { +if (ip_tunnel_encap (skb, &tunnel->encap, &protocol, &fl4) <0 ) {ip_rt_put (rt); goto tx_error; }
Signature · 1 line
2.39.2
PATCH 2/3 bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs
In reply to Christian EhrigAdd two new kfuncs that allow a BPF tc-hook, installed on an ipip device in collect-metadata mode, to control FOU encap parameters on a per-packet level. The set of kfuncs is registered with the fou module.
The bpf_skb_set_fou_encap kfunc is supposed to be used in tandem and after a successful call to the bpf_skb_set_tunnel_key bpf-helper. UDP source and destination ports can be controlled by passing a struct bpf_fou_encap. A source port of zero will auto-assign a source port. enum bpf_fou_encap_type is used to specify if the egress path should FOU or GUE encap the packet.
On the ingress path bpf_skb_get_fou_encap can be used to read UDP source and destination ports from the receiver's point of view and allows for packet multiplexing across different destination ports within a single BPF program and ipip device.
--- include/net/fou.h | 2 + net/ipv4/Makefile | 2 +- net/ipv4/fou_bpf.c | 118 ++++++++++++++++++++++++++++++++++++++++++++ net/ipv4/fou_core.c | 5 ++ 4 files changed, 126 insertions(+), 1 deletion(-) create mode 100644 net/ipv4/fou_bpf.c
include/net/fou.h +2 -0
@@ -17,4 +17,6 @@ int __fou_build_header(struct sk_buff *skb, struct ip_tunnel_encap *e,int __gue_build_header (struct sk_buff *skb,struct ip_tunnel_encap *e,u8 *protocol,__be16 *sport,int type); +int register_fou_bpf (void ); +#endif
net/ipv4/Makefile +1 -1
@@ -26,7 +26,7 @@ obj-$(CONFIG_IP_MROUTE) += ipmr.o obj-$(CONFIG_IP_MROUTE_COMMON) += ipmr_base.o obj-$(CONFIG_NET_IPIP) += ipip.o gre-y := gre_demux.o -fou-y := fou_core.o fou_nl.o +fou-y := fou_core.o fou_nl.o fou_bpf.o obj-$(CONFIG_NET_FOU) += fou.o obj-$(CONFIG_NET_IPGRE_DEMUX) += gre.o obj-$(CONFIG_NET_IPGRE) += ip_gre.o
net/ipv4/fou_bpf.c new file +118 -0
118 lines not on this page · raw message
net/ipv4/fou_core.c +5 -0
@@ -1236,10 +1236,15 @@ static int __init fou_init(void)if (ret <0 ) goto unregister; + ret =register_fou_bpf (); +if (ret <0 ) + goto kfunc_failed; + ret =ip_tunnel_encap_add_fou_ops ();if (ret ==0 )return 0 ; +kfunc_failed:genl_unregister_family (&fou_nl_family); unregister:unregister_pernet_device (&fou_net_ops);
Signature · 1 line
2.39.2
PATCH 3/3 selftests/bpf: Test FOU kfuncs for externally controlled ipip devices
In reply to Christian EhrigAdd tests for FOU and GUE encapsulation via the bpf_skb_{set,get}_fou_encap kfuncs, using ipip devices in collect-metadata mode.
These tests make sure that we can successfully set and obtain FOU and GUE encap parameters using ingress / egress BPF tc-hooks.
--- .../selftests/bpf/progs/test_tunnel_kern.c | 117 ++++++++++++++++++ tools/testing/selftests/bpf/test_tunnel.sh | 81 ++++++++++++ 2 files changed, 198 insertions(+)
tools/testing/selftests/bpf/progs/test_tunnel_kern.c +117 -0
@@ -52,6 +52,21 @@ struct vxlan_metadata {__u32 gbp; }; +struct bpf_fou_encap { +__be16 sport; +__be16 dport; +}; + +enum bpf_fou_encap_type { +FOU_BPF_ENCAP_FOU , +FOU_BPF_ENCAP_GUE , +}; + +int bpf_skb_set_fou_encap(struct __sk_buff *skb_ctx, + struct bpf_fou_encap *encap, int type) __ksym; +int bpf_skb_get_fou_encap (struct __sk_buff *skb_ctx, + struct bpf_fou_encap *encap) __ksym; + struct {__uint (type,BPF_MAP_TYPE_ARRAY );__uint (max_entries,1 );
@@ -749,6 +764,108 @@ int ipip_get_tunnel(struct __sk_buff *skb)return TC_ACT_OK ; } +SEC("tc") +int ipip_gue_set_tunnel (struct __sk_buff *skb) +{ +struct bpf_tunnel_key key = {}; +struct bpf_fou_encap encap = {}; +void *data = (void *)(long )skb->data; +struct iphdr *iph = data; +void *data_end = (void *)(long )skb->data_end; +int ret; + +if (data +sizeof (*iph) > data_end) { +log_err (1 ); +return TC_ACT_SHOT ; + } + + key.tunnel_ttl =64 ; +if (iph->protocol ==IPPROTO_ICMP ) + key.remote_ipv4 =0xac100164 ;/* 172.16.1.100 */ + + ret =bpf_skb_set_tunnel_key (skb, &key,sizeof (key),0 ); +if (ret <0 ) { +log_err (ret); +return TC_ACT_SHOT ; + } + + encap.sport =0 ; + encap.dport =bpf_htons (5555 ); + + ret =bpf_skb_set_fou_encap (skb, &encap,FOU_BPF_ENCAP_GUE ); +if (ret <0 ) { +log_err (ret); +return TC_ACT_SHOT ; + } + +return TC_ACT_OK ; +} + +SEC("tc") +int ipip_fou_set_tunnel (struct __sk_buff *skb) +{ +struct bpf_tunnel_key key = {}; +struct bpf_fou_encap encap = {}; +void *data = (void *)(long )skb->data; +struct iphdr *iph = data; +void *data_end = (void *)(long )skb->data_end; +int ret; + +if (data +sizeof (*iph) > data_end) { +log_err (1 ); +return TC_ACT_SHOT ; + } + + key.tunnel_ttl =64 ; +if (iph->protocol ==IPPROTO_ICMP ) + key.remote_ipv4 =0xac100164 ;/* 172.16.1.100 */ + + ret =bpf_skb_set_tunnel_key (skb, &key,sizeof (key),0 ); +if (ret <0 ) { +log_err (ret); +return TC_ACT_SHOT ; + } + + encap.sport =0 ; + encap.dport =bpf_htons (5555 ); + + ret =bpf_skb_set_fou_encap (skb, &encap,FOU_BPF_ENCAP_FOU ); +if (ret <0 ) { +log_err (ret); +return TC_ACT_SHOT ; + } + +return TC_ACT_OK ; +} + +SEC("tc") +int ipip_encap_get_tunnel (struct __sk_buff *skb) +{ +int ret; +struct bpf_tunnel_key key = {}; +struct bpf_fou_encap encap = {}; + + ret =bpf_skb_get_tunnel_key (skb, &key,sizeof (key),0 ); +if (ret <0 ) { +log_err (ret); +return TC_ACT_SHOT ; + } + + ret =bpf_skb_get_fou_encap (skb, &encap); +if (ret <0 ) { +log_err (ret); +return TC_ACT_SHOT ; + } + +if (bpf_ntohs (encap.dport) !=5555 ) +return TC_ACT_SHOT ; + +bpf_printk ("%d remote ip 0x%x, sport %d, dport %d\n" , ret, + key.remote_ipv4,bpf_ntohs (encap.sport), +bpf_ntohs (encap.dport)); +return TC_ACT_OK ; +} + SEC("tc")int ipip6_set_tunnel (struct __sk_buff *skb) {
tools/testing/selftests/bpf/test_tunnel.sh +81 -0
105 lines not on this page · raw message
Signature · 1 line
2.39.2
kernel test robot
build botRe: [PATCH bpf-next 2/3] bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs
In reply to Christian EhrigThank you for the patch! Perhaps something to improve:
Hi Christian,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Christian-Ehrig/ipip-ip_tunnel-sit-Add-FOU-support-for-externally-controlled-ipip-devices/20230402-033512 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/65b05e447b28d32fb0e07275dc988989f358da2c.1680379518.git.cehrig%40cloudflare.com patch subject: [PATCH bpf-next 2/3] bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs config: microblaze-randconfig-r021-20230402 (https://download.01.org/0day-ci/archive/20230402/[email protected]/config) compiler: microblaze-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/111ef54093a724110ca63e6a6279e60dd669094b
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Christian-Ehrig/ipip-ip_tunnel-sit-Add-FOU-support-for-externally-controlled-ipip-devices/20230402-033512 git checkout 111ef54093a724110ca63e6a6279e60dd669094b
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=microblaze olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=microblaze SHELL=/bin/bash net/ipv4/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> net/ipv4/fou_bpf.c:114:5: warning: no previous prototype for 'register_fou_bpf' [-Wmissing-prototypes]
114 | int register_fou_bpf(void)
| ^~~~~~~~~~~~~~~~
vim +/register_fou_bpf +114 net/ipv4/fou_bpf.c
113
> 114 int register_fou_bpf(void)
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
kernel test robot
build botRe: [PATCH bpf-next 2/3] bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs
In reply to Christian EhrigThank you for the patch! Perhaps something to improve:
Hi Christian,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Christian-Ehrig/ipip-ip_tunnel-sit-Add-FOU-support-for-externally-controlled-ipip-devices/20230402-033512 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/65b05e447b28d32fb0e07275dc988989f358da2c.1680379518.git.cehrig%40cloudflare.com patch subject: [PATCH bpf-next 2/3] bpf,fou: Add bpf_skb_{set,get}_fou_encap kfuncs config: i386-randconfig-a005 (https://download.01.org/0day-ci/archive/20230402/[email protected]/config) compiler: gcc-11 (Debian 11.3.0-8) 11.3.0
reproduce (this is a W=1 build):
# https://github.com/intel-lab-lkp/linux/commit/111ef54093a724110ca63e6a6279e60dd669094b
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Christian-Ehrig/ipip-ip_tunnel-sit-Add-FOU-support-for-externally-controlled-ipip-devices/20230402-033512 git checkout 111ef54093a724110ca63e6a6279e60dd669094b
# save the config file
mkdir build_dir && cp config build_dir/.config
make W=1 O=build_dir ARCH=i386 olddefconfig
make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash net/ipv4/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> net/ipv4/fou_bpf.c:114:5: warning: no previous prototype for 'register_fou_bpf' [-Wmissing-prototypes]
114 | int register_fou_bpf(void)
| ^~~~~~~~~~~~~~~~
vim +/register_fou_bpf +114 net/ipv4/fou_bpf.c
113
> 114 int register_fou_bpf(void)
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests