netfilter flowtable hardware offload
Subsystems (3)
Pablo Neira Ayuso
maintainercover netfilter flowtable hardware offload
Hi,
The following patchset adds hardware offload support for the flowtable infrastructure [1]. This infrastructure provides a fast datapath for the classic Linux forwarding path that users can enable through policy, eg.
table inet x {
flowtable f {
hook ingress priority 10 devices = { eth0, eth1 }
flags offload
}
chain y {
type filter hook forward priority 0; policy accept;
ip protocol tcp flow offload @f
}}
This example above enables the fastpath for TCP traffic between devices eth0 and eth1. Users can turn on the hardware offload through the 'offload' flag from the flowtable definition. If this new flag is not specified, the software flowtable datapath is used.
This patchset is composed of 4 preparation patches:
#1 Move pointer to conntrack object to the flow_offload structure. #2 Remove useless union from the flow_offload structure.
#3 Remove superfluous flow_offload_entry structure.
#4 Detach routing information from the flow_offload object to leave
room to extend this infrastructure, eg. accelerate bridge forwarding.
And 2 patches to add the hardware offload control and data planes:
#5 Add the netlink control plane and the interface to set up the flowtable
hardware offload. This includes a new NFTA_FLOWTABLE_FLAGS netlink attribute to convey the optional NF_FLOWTABLE_HW_OFFLOAD flag.
#6 Add the hardware offload datapath: This code uses the flow_offload
API available at net/core/flow_offload.h to represent the flow through two flow_rule objects to configure an exact 5-tuple matching on each direction plus the corresponding forwarding actions, that is, the MAC address, NAT and checksum updates; and port redirection in order to configure the hardware datapath. This patch only supports for IPv4 support and statistics collection for flow aging as an initial step.
This patchset introduces a new flow_block callback type that needs to be set up to configure the flowtable hardware offload.
The first client of this infrastructure follows up after this batch. I would like to thank Mellanox for developing the first upstream driver to use this infrastructure.
Please, apply.
[1] Documentation/networking/nf_flowtable.txt
Pablo Neira Ayuso (6):
netfilter: nf_flow_table: move conntrack object to struct flow_offload
netfilter: nf_flow_table: remove union from flow_offload structure
netfilter: nf_flowtable: remove flow_offload_entry structure
netfilter: nf_flow_table: detach routing information from flow description
netfilter: nf_tables: add flowtable offload control plane
netfilter: nf_flow_table: hardware offload support
include/linux/netdevice.h | 1 + include/net/netfilter/nf_flow_table.h | 60 ++- include/uapi/linux/netfilter/nf_tables.h | 2 + net/ipv4/netfilter/nf_flow_table_ipv4.c | 2 + net/ipv6/netfilter/nf_flow_table_ipv6.c | 2 + net/netfilter/Makefile | 3 +- net/netfilter/nf_flow_table_core.c | 173 ++++--- net/netfilter/nf_flow_table_inet.c | 2 + net/netfilter/nf_flow_table_offload.c | 758 +++++++++++++++++++++++++++++++ net/netfilter/nf_tables_api.c | 21 +- net/netfilter/nft_flow_offload.c | 5 +- 11 files changed, 955 insertions(+), 74 deletions(-) create mode 100644 net/netfilter/nf_flow_table_offload.c
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 1/6 netfilter: nf_flow_table: move conntrack object to struct flow_offload
Simplify this code by storing the pointer to conntrack object in the flow_offload structure.
--- include/net/netfilter/nf_flow_table.h | 1 + net/netfilter/nf_flow_table_core.c | 35 +++++++++++------------------------ 2 files changed, 12 insertions(+), 24 deletions(-)
include/net/netfilter/nf_flow_table.h +1 -0
@@ -72,6 +72,7 @@ struct flow_offload_tuple_rhash {struct flow_offload {struct flow_offload_tuple_rhash tuplehash[FLOW_OFFLOAD_DIR_MAX ]; +struct nf_conn *ct;u32 flags;union {/* Your private driver data here. */
net/netfilter/nf_flow_table_core.c +11 -24
@@ -16,7 +16,6 @@struct flow_offload_entry {struct flow_offload flow; -struct nf_conn *ct;struct rcu_head rcu_head; };
@@ -79,7 +78,7 @@ flow_offload_alloc(struct nf_conn *ct, struct nf_flow_route *route)if (!dst_hold_safe (route->tuple[FLOW_OFFLOAD_DIR_REPLY ].dst)) goto err_dst_cache_reply; - entry->ct = ct; + flow->ct = ct;flow_offload_fill_dir (flow, ct, route,FLOW_OFFLOAD_DIR_ORIGINAL );flow_offload_fill_dir (flow, ct, route,FLOW_OFFLOAD_DIR_REPLY );
@@ -158,8 +157,8 @@ void flow_offload_free(struct flow_offload *flow)dst_release (flow->tuplehash[FLOW_OFFLOAD_DIR_REPLY ].tuple.dst_cache); e = container_of(flow, struct flow_offload_entry, flow);if (flow->flags &FLOW_OFFLOAD_DYING ) -nf_ct_delete (e->ct,0 ,0 ); -nf_ct_put (e->ct); +nf_ct_delete (flow->ct,0 ,0 ); +nf_ct_put (flow->ct);kfree_rcu (e, rcu_head); }EXPORT_SYMBOL_GPL (flow_offload_free);
@@ -232,8 +231,6 @@ static inline bool nf_flow_has_expired(const struct flow_offload *flow)static void flow_offload_del (struct nf_flowtable *flow_table,struct flow_offload *flow) { -struct flow_offload_entry *e; -rhashtable_remove_fast (&flow_table->rhashtable, &flow->tuplehash[FLOW_OFFLOAD_DIR_ORIGINAL ].node, nf_flow_offload_rhash_params);
@@ -241,25 +238,21 @@ static void flow_offload_del(struct nf_flowtable *flow_table, &flow->tuplehash[FLOW_OFFLOAD_DIR_REPLY ].node, nf_flow_offload_rhash_params); - e = container_of(flow, struct flow_offload_entry, flow); -clear_bit (IPS_OFFLOAD_BIT , &e->ct->status); +clear_bit (IPS_OFFLOAD_BIT , &flow->ct->status);if (nf_flow_has_expired (flow)) -flow_offload_fixup_ct (e->ct); +flow_offload_fixup_ct (flow->ct);else if (flow->flags &FLOW_OFFLOAD_TEARDOWN ) -flow_offload_fixup_ct_timeout (e->ct); +flow_offload_fixup_ct_timeout (flow->ct);flow_offload_free (flow); }void flow_offload_teardown (struct flow_offload *flow) { -struct flow_offload_entry *e; - flow->flags |=FLOW_OFFLOAD_TEARDOWN ; - e = container_of(flow, struct flow_offload_entry, flow); -flow_offload_fixup_ct_state (e->ct); +flow_offload_fixup_ct_state (flow->ct); }EXPORT_SYMBOL_GPL (flow_offload_teardown);
@@ -269,7 +262,6 @@ flow_offload_lookup(struct nf_flowtable *flow_table, {struct flow_offload_tuple_rhash *tuplehash;struct flow_offload *flow; -struct flow_offload_entry *e;int dir; tuplehash = rhashtable_lookup(&flow_table->rhashtable, tuple,
@@ -282,8 +274,7 @@ flow_offload_lookup(struct nf_flowtable *flow_table,if (flow->flags & (FLOW_OFFLOAD_DYING |FLOW_OFFLOAD_TEARDOWN ))return NULL ; - e = container_of(flow, struct flow_offload_entry, flow); -if (unlikely (nf_ct_is_dying (e->ct))) +if (unlikely (nf_ct_is_dying (flow->ct)))return NULL ;return tuplehash;
@@ -327,10 +318,8 @@ nf_flow_table_iterate(struct nf_flowtable *flow_table,static void nf_flow_offload_gc_step (struct flow_offload *flow,void *data) {struct nf_flowtable *flow_table = data; -struct flow_offload_entry *e; - e = container_of(flow, struct flow_offload_entry, flow); -if (nf_flow_has_expired (flow) ||nf_ct_is_dying (e->ct) || +if (nf_flow_has_expired (flow) ||nf_ct_is_dying (flow->ct) || (flow->flags & (FLOW_OFFLOAD_DYING |FLOW_OFFLOAD_TEARDOWN )))flow_offload_del (flow_table, flow); }
@@ -485,15 +474,13 @@ EXPORT_SYMBOL_GPL(nf_flow_table_init);static void nf_flow_table_do_cleanup (struct flow_offload *flow,void *data) {struct net_device *dev = data; -struct flow_offload_entry *e; - - e = container_of(flow, struct flow_offload_entry, flow);if (!dev) {flow_offload_teardown (flow);return ; } -if (net_eq (nf_ct_net (e->ct),dev_net (dev)) && + +if (net_eq (nf_ct_net (flow->ct),dev_net (dev)) && (flow->tuplehash[0 ].tuple.iifidx == dev->ifindex || flow->tuplehash[1 ].tuple.iifidx == dev->ifindex))flow_offload_dead (flow);
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 2/6 netfilter: nf_flow_table: remove union from flow_offload structure
In reply to Pablo Neira AyusoDrivers do not have access to the flow_offload structure, hence remove this union from this flow_offload object as well as the original comment on top of it.
--- include/net/netfilter/nf_flow_table.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
include/net/netfilter/nf_flow_table.h +1 -4
@@ -74,10 +74,7 @@ struct flow_offload {struct flow_offload_tuple_rhash tuplehash[FLOW_OFFLOAD_DIR_MAX ];struct nf_conn *ct;u32 flags; -union { -/* Your private driver data here. */ -u32 timeout; - }; +u32 timeout; };#define NF_FLOW_TIMEOUT (30 * HZ)
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 3/6 netfilter: nf_flowtable: remove flow_offload_entry structure
In reply to Pablo Neira AyusoMove rcu_head to struct flow_offload, then remove the flow_offload_entry structure definition.
--- include/net/netfilter/nf_flow_table.h | 1 + net/netfilter/nf_flow_table_core.c | 19 ++++--------------- 2 files changed, 5 insertions(+), 15 deletions(-)
include/net/netfilter/nf_flow_table.h +1 -0
@@ -75,6 +75,7 @@ struct flow_offload {struct nf_conn *ct;u32 flags;u32 timeout; +struct rcu_head rcu_head; };#define NF_FLOW_TIMEOUT (30 * HZ)
net/netfilter/nf_flow_table_core.c +4 -15
@@ -14,11 +14,6 @@#include <net/netfilter/nf_conntrack_l4proto.h> #include <net/netfilter/nf_conntrack_tuple.h> -struct flow_offload_entry { -struct flow_offload flow; -struct rcu_head rcu_head; -}; -static DEFINE_MUTEX (flowtable_lock);static LIST_HEAD (flowtables);
@@ -59,19 +54,16 @@ flow_offload_fill_dir(struct flow_offload *flow, struct nf_conn *ct,struct flow_offload *flow_offload_alloc (struct nf_conn *ct,struct nf_flow_route *route) { -struct flow_offload_entry *entry;struct flow_offload *flow;if (unlikely (nf_ct_is_dying (ct) || !atomic_inc_not_zero (&ct->ct_general.use)))return NULL ; - entry =kzalloc (sizeof (*entry),GFP_ATOMIC ); -if (!entry) + flow =kzalloc (sizeof (*flow),GFP_ATOMIC ); +if (!flow) goto err_ct_refcnt; - flow = &entry->flow; -if (!dst_hold_safe (route->tuple[FLOW_OFFLOAD_DIR_ORIGINAL ].dst)) goto err_dst_cache_original;
@@ -93,7 +85,7 @@ flow_offload_alloc(struct nf_conn *ct, struct nf_flow_route *route) err_dst_cache_reply:dst_release (route->tuple[FLOW_OFFLOAD_DIR_ORIGINAL ].dst); err_dst_cache_original: -kfree (entry); +kfree (flow); err_ct_refcnt:nf_ct_put (ct);
@@ -151,15 +143,12 @@ static void flow_offload_fixup_ct(struct nf_conn *ct)void flow_offload_free (struct flow_offload *flow) { -struct flow_offload_entry *e; -dst_release (flow->tuplehash[FLOW_OFFLOAD_DIR_ORIGINAL ].tuple.dst_cache);dst_release (flow->tuplehash[FLOW_OFFLOAD_DIR_REPLY ].tuple.dst_cache); - e = container_of(flow, struct flow_offload_entry, flow);if (flow->flags &FLOW_OFFLOAD_DYING )nf_ct_delete (flow->ct,0 ,0 );nf_ct_put (flow->ct); -kfree_rcu (e, rcu_head); +kfree_rcu (flow, rcu_head); }EXPORT_SYMBOL_GPL (flow_offload_free);
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 4/6 netfilter: nf_flow_table: detach routing information from flow description
In reply to Pablo Neira AyusoThis patch adds the infrastructure to support for flow entry types. The initial type is NF_FLOW_OFFLOAD_ROUTE that stores the routing information into the flow entry to define a fastpath for the classic forwarding path.
--- include/net/netfilter/nf_flow_table.h | 14 ++++-- net/netfilter/nf_flow_table_core.c | 88 ++++++++++++++++++++++++++--------- net/netfilter/nft_flow_offload.c | 5 +- 3 files changed, 80 insertions(+), 27 deletions(-)
include/net/netfilter/nf_flow_table.h +11 -3
@@ -70,10 +70,16 @@ struct flow_offload_tuple_rhash {#define FLOW_OFFLOAD_DYING 0x4#define FLOW_OFFLOAD_TEARDOWN 0x8 +enum flow_offload_type { +NF_FLOW_OFFLOAD_UNSPEC =0 , +NF_FLOW_OFFLOAD_ROUTE , +}; +struct flow_offload {struct flow_offload_tuple_rhash tuplehash[FLOW_OFFLOAD_DIR_MAX ];struct nf_conn *ct; -u32 flags; +u16 flags; +u16 type;u32 timeout;struct rcu_head rcu_head; };
@@ -86,10 +92,12 @@ struct nf_flow_route { } tuple[FLOW_OFFLOAD_DIR_MAX ]; }; -struct flow_offload *flow_offload_alloc (struct nf_conn *ct, -struct nf_flow_route *route); +struct flow_offload *flow_offload_alloc (struct nf_conn *ct);void flow_offload_free (struct flow_offload *flow); +int flow_offload_route_init (struct flow_offload *flow, +const struct nf_flow_route *route); +int flow_offload_add (struct nf_flowtable *flow_table,struct flow_offload *flow);struct flow_offload_tuple_rhash *flow_offload_lookup (struct nf_flowtable *flow_table,struct flow_offload_tuple *tuple);
net/netfilter/nf_flow_table_core.c +65 -23
145 lines not on this page · raw message
net/netfilter/nft_flow_offload.c +4 -1
@@ -115,10 +115,13 @@ static void nft_flow_offload_eval(const struct nft_expr *expr,if (nft_flow_route (pkt, ct, &route, dir) <0 ) goto err_flow_route; - flow =flow_offload_alloc (ct, &route); + flow =flow_offload_alloc (ct);if (!flow) goto err_flow_alloc; +if (flow_offload_route_init (flow, &route) <0 ) + goto err_flow_add; +if (tcph) { ct->proto.tcp.seen[0 ].flags |=IP_CT_TCP_FLAG_BE_LIBERAL ; ct->proto.tcp.seen[1 ].flags |=IP_CT_TCP_FLAG_BE_LIBERAL ;
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 5/6 netfilter: nf_tables: add flowtable offload control plane
In reply to Pablo Neira AyusoThis patch adds the NFTA_FLOWTABLE_FLAGS attribute that allows users to specify the NF_FLOWTABLE_HW_OFFLOAD flag. This patch also adds a new setup interface for the flowtable type to perform the flowtable offload block callback configuration.
--- include/net/netfilter/nf_flow_table.h | 18 ++++++++++++++++++ include/uapi/linux/netfilter/nf_tables.h | 2 ++ net/ipv4/netfilter/nf_flow_table_ipv4.c | 1 + net/ipv6/netfilter/nf_flow_table_ipv6.c | 1 + net/netfilter/nf_flow_table_inet.c | 1 + net/netfilter/nf_tables_api.c | 21 +++++++++++++++++++-- 6 files changed, 42 insertions(+), 2 deletions(-)
include/net/netfilter/nf_flow_table.h +18 -0
@@ -8,6 +8,7 @@#include <linux/rcupdate.h> #include <linux/netfilter.h> #include <linux/netfilter/nf_conntrack_tuple_common.h> +#include <net/flow_offload.h> #include <net/dst.h> struct nf_flowtable ;
@@ -16,17 +17,27 @@ struct nf_flowtable_type {struct list_head list;int family;int (*init)(struct nf_flowtable *ft); +int (*setup)(struct nf_flowtable *ft, +struct net_device *dev, +enum flow_block_command cmd);void (*free)(struct nf_flowtable *ft);nf_hookfn *hook;struct module *owner; }; +enum nf_flowtable_flags { +NF_FLOWTABLE_HW_OFFLOAD =0x1 , +}; +struct nf_flowtable {struct list_head list;struct rhashtable rhashtable;int priority;const struct nf_flowtable_type *type;struct delayed_work gc_work; +unsigned int flags; +struct flow_block flow_block; +possible_net_t net; };enum flow_offload_tuple_dir {
@@ -131,4 +142,11 @@ unsigned int nf_flow_offload_ipv6_hook(void *priv, struct sk_buff *skb,#define MODULE_ALIAS_NF_FLOWTABLE (family) \ MODULE_ALIAS("nf-flowtable-" __stringify(family)) +static inline int nf_flow_table_offload_setup (struct nf_flowtable *flowtable, +struct net_device *dev, +enum flow_block_command cmd) +{ +return 0 ; +} +#endif /* _NF_FLOW_TABLE_H */
include/uapi/linux/netfilter/nf_tables.h +2 -0
@@ -1518,6 +1518,7 @@ enum nft_object_attributes {* @NFTA_FLOWTABLE_HOOK: netfilter hook configuration(NLA_U32) * @NFTA_FLOWTABLE_USE: number of references to this flow table (NLA_U32) * @NFTA_FLOWTABLE_HANDLE: object handle (NLA_U64) +* @NFTA_FLOWTABLE_FLAGS: flags (NLA_U32) */ enum nft_flowtable_attributes {NFTA_FLOWTABLE_UNSPEC ,
@@ -1527,6 +1528,7 @@ enum nft_flowtable_attributes {NFTA_FLOWTABLE_USE ,NFTA_FLOWTABLE_HANDLE ,NFTA_FLOWTABLE_PAD , +NFTA_FLOWTABLE_FLAGS , __NFTA_FLOWTABLE_MAX };#define NFTA_FLOWTABLE_MAX (__NFTA_FLOWTABLE_MAX - 1)
net/ipv4/netfilter/nf_flow_table_ipv4.c +1 -0
@@ -9,6 +9,7 @@static struct nf_flowtable_type flowtable_ipv4 = { .family =NFPROTO_IPV4 , .init = nf_flow_table_init, + .setup = nf_flow_table_offload_setup, .free = nf_flow_table_free, .hook = nf_flow_offload_ip_hook, .owner =THIS_MODULE ,
net/ipv6/netfilter/nf_flow_table_ipv6.c +1 -0
@@ -10,6 +10,7 @@static struct nf_flowtable_type flowtable_ipv6 = { .family =NFPROTO_IPV6 , .init = nf_flow_table_init, + .setup = nf_flow_table_offload_setup, .free = nf_flow_table_free, .hook = nf_flow_offload_ipv6_hook, .owner =THIS_MODULE ,
net/netfilter/nf_flow_table_inet.c +1 -0
@@ -24,6 +24,7 @@ nf_flow_offload_inet_hook(void *priv, struct sk_buff *skb,static struct nf_flowtable_type flowtable_inet = { .family =NFPROTO_INET , .init = nf_flow_table_init, + .setup = nf_flow_table_offload_setup, .free = nf_flow_table_free, .hook = nf_flow_offload_inet_hook, .owner =THIS_MODULE ,
net/netfilter/nf_tables_api.c +19 -2
@@ -5835,6 +5835,7 @@ static const struct nla_policy nft_flowtable_policy[NFTA_FLOWTABLE_MAX + 1] = { .len = NFT_NAME_MAXLEN - 1 }, [NFTA_FLOWTABLE_HOOK] = { .type = NLA_NESTED }, [NFTA_FLOWTABLE_HANDLE] = { .type = NLA_U64 }, + [NFTA_FLOWTABLE_FLAGS] = { .type = NLA_U32 }, }; struct nft_flowtable *nft_flowtable_lookup(const struct nft_table *table,
@@ -5968,8 +5969,11 @@ static void nft_unregister_flowtable_net_hooks(struct net *net, {struct nft_hook *hook; -list_for_each_entry (hook, &flowtable->hook_list, list) +list_for_each_entry (hook, &flowtable->hook_list, list) {nf_unregister_net_hook (net, &hook->ops); + flowtable->data.type->setup (&flowtable->data, hook->ops.dev, +FLOW_BLOCK_UNBIND ); + } } static int nft_register_flowtable_net_hooks(struct net *net,
@@ -5991,6 +5995,8 @@ static int nft_register_flowtable_net_hooks(struct net *net, } } + flowtable->data.type->setup (&flowtable->data, hook->ops.dev, +FLOW_BLOCK_BIND ); err =nf_register_net_hook (net, &hook->ops);if (err <0 ) goto err_unregister_net_hooks;
@@ -6006,6 +6012,8 @@ static int nft_register_flowtable_net_hooks(struct net *net,break ;nf_unregister_net_hook (net, &hook->ops); + flowtable->data.type->setup (&flowtable->data, hook->ops.dev, +FLOW_BLOCK_UNBIND );list_del_rcu (&hook->list);kfree_rcu (hook, rcu); }
@@ -6080,6 +6088,14 @@ static int nf_tables_newflowtable(struct net *net, struct sock *nlsk, goto err2; } +if (nla[NFTA_FLOWTABLE_FLAGS ]) { + flowtable->data.flags = +ntohl (nla_get_be32 (nla[NFTA_FLOWTABLE_FLAGS ])); +if (flowtable->data.flags & ~NF_FLOWTABLE_HW_OFFLOAD ) + goto err3; + } + +write_pnet (&flowtable->data.net, net); flowtable->data.type = type; err = type->init (&flowtable->data);if (err <0 )
@@ -6191,7 +6207,8 @@ static int nf_tables_fill_flowtable_info(struct sk_buff *skb, struct net *net,nla_put_string (skb,NFTA_FLOWTABLE_NAME , flowtable->name) ||nla_put_be32 (skb,NFTA_FLOWTABLE_USE ,htonl (flowtable->use)) ||nla_put_be64 (skb,NFTA_FLOWTABLE_HANDLE ,cpu_to_be64 (flowtable->handle), - NFTA_FLOWTABLE_PAD)) +NFTA_FLOWTABLE_PAD ) || + nla_put_be32(skb, NFTA_FLOWTABLE_FLAGS, htonl(flowtable->data.flags))) goto nla_put_failure; nest =nla_nest_start_noflag (skb,NFTA_FLOWTABLE_HOOK );
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 6/6 netfilter: nf_flow_table: hardware offload support
In reply to Pablo Neira AyusoThis patch adds the dataplane hardware offload to the flowtable infrastructure. Three new flags represent the hardware state of this flow:
* FLOW_OFFLOAD_HW: This flow entry resides in the hardware.
* FLOW_OFFLOAD_HW_DYING: This flow entry has been scheduled to be remove from hardware. This might be triggered by either packet path (via TCP RST/FIN packet) or via aging.
* FLOW_OFFLOAD_HW_DEAD: This flow entry has been already removed from the hardware, the software garbage collector can remove it from the software flowtable.
This patch supports for:
* IPv4 only.
* Aging via FLOW_CLS_STATS, no packet and byte counter synchronization at this stage.
This patch also adds the action callback that specifies how to convert the flow entry into the flow_rule object that is passed to the driver.
--- include/linux/netdevice.h | 1 + include/net/netfilter/nf_flow_table.h | 33 +- net/ipv4/netfilter/nf_flow_table_ipv4.c | 1 + net/ipv6/netfilter/nf_flow_table_ipv6.c | 1 + net/netfilter/Makefile | 3 +- net/netfilter/nf_flow_table_core.c | 33 +- net/netfilter/nf_flow_table_inet.c | 1 + net/netfilter/nf_flow_table_offload.c | 758 ++++++++++++++++++++++++++++++++ 8 files changed, 822 insertions(+), 9 deletions(-) create mode 100644 net/netfilter/nf_flow_table_offload.c
include/linux/netdevice.h +1 -0
@@ -848,6 +848,7 @@ enum tc_setup_type {TC_SETUP_ROOT_QDISC ,TC_SETUP_QDISC_GRED ,TC_SETUP_QDISC_TAPRIO , + TC_SETUP_FT, };/* These structures hold the attributes of bpf state that are being passed
include/net/netfilter/nf_flow_table.h +27 -6
@@ -12,6 +12,9 @@#include <net/dst.h> struct nf_flowtable ; +struct nf_flow_rule ; +struct flow_offload ; +enum flow_offload_tuple_dir ;struct nf_flowtable_type {struct list_head list;
@@ -20,6 +23,10 @@ struct nf_flowtable_type {int (*setup)(struct nf_flowtable *ft,struct net_device *dev,enum flow_block_command cmd); +int (*action)(struct net *net, +const struct flow_offload *flow, +enum flow_offload_tuple_dir dir, +struct nf_flow_rule *flow_rule);void (*free)(struct nf_flowtable *ft);nf_hookfn *hook;struct module *owner;
@@ -80,6 +87,9 @@ struct flow_offload_tuple_rhash {#define FLOW_OFFLOAD_DNAT 0x2#define FLOW_OFFLOAD_DYING 0x4#define FLOW_OFFLOAD_TEARDOWN 0x8 +#define FLOW_OFFLOAD_HW 0x10 +#define FLOW_OFFLOAD_HW_DYING 0x20 +#define FLOW_OFFLOAD_HW_DEAD 0x40enum flow_offload_type {NF_FLOW_OFFLOAD_UNSPEC =0 ,
@@ -142,11 +152,22 @@ unsigned int nf_flow_offload_ipv6_hook(void *priv, struct sk_buff *skb,#define MODULE_ALIAS_NF_FLOWTABLE (family) \ MODULE_ALIAS("nf-flowtable-" __stringify(family)) -static inline int nf_flow_table_offload_setup (struct nf_flowtable *flowtable, -struct net_device *dev, -enum flow_block_command cmd) -{ -return 0 ; -} +void nf_flow_offload_add (struct nf_flowtable *flowtable, +struct flow_offload *flow); +void nf_flow_offload_del (struct nf_flowtable *flowtable, +struct flow_offload *flow); +void nf_flow_offload_stats (struct nf_flowtable *flowtable, +struct flow_offload *flow); + +void nf_flow_table_offload_flush (struct nf_flowtable *flowtable); +int nf_flow_table_offload_setup (struct nf_flowtable *flowtable, +struct net_device *dev, +enum flow_block_command cmd); +int nf_flow_rule_route (struct net *net,const struct flow_offload *flow, +enum flow_offload_tuple_dir dir, +struct nf_flow_rule *flow_rule); + +int nf_flow_table_offload_init (void ); +void nf_flow_table_offload_exit (void );#endif /* _NF_FLOW_TABLE_H */
net/ipv4/netfilter/nf_flow_table_ipv4.c +1 -0
@@ -10,6 +10,7 @@ static struct nf_flowtable_type flowtable_ipv4 = { .family = NFPROTO_IPV4, .init = nf_flow_table_init, .setup = nf_flow_table_offload_setup, + .action = nf_flow_rule_route, .free = nf_flow_table_free, .hook = nf_flow_offload_ip_hook, .owner = THIS_MODULE,
net/ipv6/netfilter/nf_flow_table_ipv6.c +1 -0
@@ -11,6 +11,7 @@ static struct nf_flowtable_type flowtable_ipv6 = { .family = NFPROTO_IPV6, .init = nf_flow_table_init, .setup = nf_flow_table_offload_setup, + .action = nf_flow_rule_route, .free = nf_flow_table_free, .hook = nf_flow_offload_ipv6_hook, .owner = THIS_MODULE,
net/netfilter/Makefile +2 -1
@@ -120,7 +120,8 @@ obj-$(CONFIG_NFT_FWD_NETDEV) += nft_fwd_netdev.o # flow table infrastructure obj-$(CONFIG_NF_FLOW_TABLE) += nf_flow_table.o -nf_flow_table-objs := nf_flow_table_core.o nf_flow_table_ip.o +nf_flow_table-objs := nf_flow_table_core.o nf_flow_table_ip.o \ + nf_flow_table_offload.o obj-$(CONFIG_NF_FLOW_TABLE_INET) += nf_flow_table_inet.o
net/netfilter/nf_flow_table_core.c +31 -2
@@ -250,6 +250,9 @@ int flow_offload_add(struct nf_flowtable *flow_table, struct flow_offload *flow)return err; } +if (flow_table->flags &NF_FLOWTABLE_HW_OFFLOAD ) +nf_flow_offload_add (flow_table, flow); +return 0 ; }EXPORT_SYMBOL_GPL (flow_offload_add);
@@ -350,9 +353,20 @@ static void nf_flow_offload_gc_step(struct flow_offload *flow, void *data) {struct nf_flowtable *flow_table = data; +if (flow->flags &FLOW_OFFLOAD_HW ) +nf_flow_offload_stats (flow_table, flow); +if (nf_flow_has_expired (flow) ||nf_ct_is_dying (flow->ct) || - (flow->flags & (FLOW_OFFLOAD_DYING |FLOW_OFFLOAD_TEARDOWN ))) -flow_offload_del (flow_table, flow); + (flow->flags & (FLOW_OFFLOAD_DYING |FLOW_OFFLOAD_TEARDOWN ))) { +if (flow->flags &FLOW_OFFLOAD_HW ) { +if (!(flow->flags &FLOW_OFFLOAD_HW_DYING )) +nf_flow_offload_del (flow_table, flow); +else if (flow->flags &FLOW_OFFLOAD_HW_DEAD ) +flow_offload_del (flow_table, flow); + }else { +flow_offload_del (flow_table, flow); + } + } }static void nf_flow_offload_work_gc (struct work_struct *work)
@@ -485,6 +499,7 @@ int nf_flow_table_init(struct nf_flowtable *flowtable)int err;INIT_DEFERRABLE_WORK (&flowtable->gc_work, nf_flow_offload_work_gc); +flow_block_init (&flowtable->flow_block); err =rhashtable_init (&flowtable->rhashtable, &nf_flow_offload_rhash_params);
@@ -520,6 +535,7 @@ static void nf_flow_table_do_cleanup(struct flow_offload *flow, void *data)static void nf_flow_table_iterate_cleanup (struct nf_flowtable *flowtable,struct net_device *dev) { +nf_flow_table_offload_flush (flowtable);nf_flow_table_iterate (flowtable, nf_flow_table_do_cleanup, dev);flush_delayed_work (&flowtable->gc_work); }
@@ -547,5 +563,18 @@ void nf_flow_table_free(struct nf_flowtable *flow_table) }EXPORT_SYMBOL_GPL (nf_flow_table_free); +static int __init nf_flow_table_module_init (void ) +{ +return nf_flow_table_offload_init (); +} + +static void __exit nf_flow_table_module_exit(void) +{ +nf_flow_table_offload_exit (); +} + +module_init (nf_flow_table_module_init); +module_exit (nf_flow_table_module_exit); +MODULE_LICENSE ("GPL" );MODULE_AUTHOR ("Pablo Neira Ayuso <[email protected]>" );
net/netfilter/nf_flow_table_inet.c +1 -0
@@ -25,6 +25,7 @@ static struct nf_flowtable_type flowtable_inet = { .family = NFPROTO_INET, .init = nf_flow_table_init, .setup = nf_flow_table_offload_setup, + .action = nf_flow_rule_route, .free = nf_flow_table_free, .hook = nf_flow_offload_inet_hook, .owner = THIS_MODULE,
net/netfilter/nf_flow_table_offload.c new file +758 -0
758 lines not on this page · raw message
Signature · 1 line
2.11.0
kbuild test robot
build botRe: [PATCH net-next 6/6] netfilter: nf_flow_table: hardware offload support
I love your patch! Perhaps something to improve:
Hi Pablo,
I love your patch! Perhaps something to improve:
[auto build test WARNING on net-next/master]
[also build test WARNING on next-20191112]
[cannot apply to v5.4-rc7]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Pablo-Neira-Ayuso/netfilter-flowtable-hardware-offload/20191113-052213 base: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git ca22d6977b9b4ab0fd2e7909b57e32ba5b95046f config: mips-allmodconfig (attached as .config) compiler: mips-linux-gcc (GCC) 7.4.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.4.0 make.cross ARCH=mips
If you fix the issue, kindly add following tag
All warnings (new ones prefixed by >>):
net/netfilter/nf_flow_table_offload.c: In function 'nf_flow_rule_match':
>> net/netfilter/nf_flow_table_offload.c:80:21: warning: large integer implicitly truncated to unsigned type [-Woverflow]
mask->tcp.flags = TCP_FLAG_RST | TCP_FLAG_FIN;
^~~~~~~~~~~~vim +80 net/netfilter/nf_flow_table_offload.c
46
47 #define NF_FLOW_DISSECTOR(__match, __type, __field) \
48 (__match)->dissector.offset[__type] = \
49 offsetof(struct nf_flow_key, __field)
50
51 static int nf_flow_rule_match(struct nf_flow_match *match,
52 const struct flow_offload_tuple *tuple)
53 {
54 struct nf_flow_key *mask = &match->mask;
55 struct nf_flow_key *key = &match->key;
56
57 NF_FLOW_DISSECTOR(match, FLOW_DISSECTOR_KEY_CONTROL, control);
58 NF_FLOW_DISSECTOR(match, FLOW_DISSECTOR_KEY_BASIC, basic);
59 NF_FLOW_DISSECTOR(match, FLOW_DISSECTOR_KEY_IPV4_ADDRS, ipv4);
60 NF_FLOW_DISSECTOR(match, FLOW_DISSECTOR_KEY_TCP, tcp);
61 NF_FLOW_DISSECTOR(match, FLOW_DISSECTOR_KEY_PORTS, tp);
62
63 switch (tuple->l3proto) {
64 case AF_INET:
65 key->control.addr_type = FLOW_DISSECTOR_KEY_IPV4_ADDRS;
66 key->basic.n_proto = htons(ETH_P_IP);
67 key->ipv4.src = tuple->src_v4.s_addr;
68 mask->ipv4.src = 0xffffffff;
69 key->ipv4.dst = tuple->dst_v4.s_addr;
70 mask->ipv4.dst = 0xffffffff;
71 break;
72 default:
73 return -EOPNOTSUPP;
74 }
75 mask->basic.n_proto = 0xffff;
76
77 switch (tuple->l4proto) {
78 case IPPROTO_TCP:
79 key->tcp.flags = 0;
> 80 mask->tcp.flags = TCP_FLAG_RST | TCP_FLAG_FIN;
81 match->dissector.used_keys |= BIT(FLOW_DISSECTOR_KEY_TCP);
82 break;
83 case IPPROTO_UDP:
84 break;
85 default:
86 return -EOPNOTSUPP;
87 }
88
89 key->basic.ip_proto = tuple->l4proto;
90 mask->basic.ip_proto = 0xff;
91
92 key->tp.src = tuple->src_port;
93 mask->tp.src = 0xffff;
94 key->tp.dst = tuple->dst_port;
95 mask->tp.dst = 0xffff;
96
97 match->dissector.used_keys |= BIT(FLOW_DISSECTOR_KEY_CONTROL) |
98 BIT(FLOW_DISSECTOR_KEY_BASIC) |
99 BIT(FLOW_DISSECTOR_KEY_IPV4_ADDRS) | 100 BIT(FLOW_DISSECTOR_KEY_PORTS);
101 return 0;
102 }
103
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org Intel Corporation
Re: [PATCH net-next 6/6] netfilter: nf_flow_table: hardware offload support
In reply to Pablo Neira AyusoQuoted 22 lines · Pablo Neira Ayuso
This patch adds the dataplane hardware offload to the flowtable infrastructure. Three new flags represent the hardware state of this flow:
*
FLOW_OFFLOAD_HW: This flow entry resides in the hardware.
*FLOW_OFFLOAD_HW_DYING: This flow entry has been scheduled to be remove from hardware. This might be triggered by either packet path (via TCP RST/FIN packet) or via aging.
*FLOW_OFFLOAD_HW_DEAD: This flow entry has been already removed from the hardware, the software garbage collector can remove it from the software flowtable.This patch supports for:
* IPv4 only.
* Aging viaFLOW_CLS_STATS, no packet and byte counter synchronization at this stage.This patch also adds the action callback that specifies how to convert the flow entry into the
flow_ruleobject that is passed to the driver.Signed-off-by Pablo Neira Ayuso <[email protected]>
<snip>
Quoted 5 lines
net/netfilter/nf_flow_table_offload.c new file +5
-0+static int nf_flow_rule_match (struct nf_flow_match *match, +const struct flow_offload_tuple *tuple) +{ +struct nf_flow_key *mask = &match->mask; +struct nf_flow_key *key = &match->key;
net/netfilter/nf_flow_table_offload.c line 65
Quoted 10 lines
net/netfilter/nf_flow_table_offload.c new file +10
-0+ +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_CONTROL , control); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_BASIC , basic); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_IPV4_ADDRS , ipv4); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_TCP , tcp); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_PORTS , tp); + +switch (tuple->l3proto) { +case AF_INET : + key->control.addr_type =FLOW_DISSECTOR_KEY_IPV4_ADDRS ;
Is it intentional that mask->control.addr_type never gets set?
-ed
Re: [PATCH net-next 6/6] netfilter: nf_flow_table: hardware offload support
Quoted 24 lines · Edward Cree
Quoted 22 lines · Pablo Neira Ayuso
This patch adds the dataplane hardware offload to the flowtable infrastructure. Three new flags represent the hardware state of this flow:
*
FLOW_OFFLOAD_HW: This flow entry resides in the hardware.
*FLOW_OFFLOAD_HW_DYING: This flow entry has been scheduled to be remove from hardware. This might be triggered by either packet path (via TCP RST/FIN packet) or via aging.
*FLOW_OFFLOAD_HW_DEAD: This flow entry has been already removed from the hardware, the software garbage collector can remove it from the software flowtable.This patch supports for:
* IPv4 only.
* Aging viaFLOW_CLS_STATS, no packet and byte counter synchronization at this stage.This patch also adds the action callback that specifies how to convert the flow entry into the
flow_ruleobject that is passed to the driver.Signed-off-by Pablo Neira Ayuso <[email protected]><snip>
Quoted 16 lines
Quoted 5 lines
net/netfilter/nf_flow_table_offload.c new file +5
-0+static int nf_flow_rule_match (struct nf_flow_match *match, +const struct flow_offload_tuple *tuple) +{ +struct nf_flow_key *mask = &match->mask; +struct nf_flow_key *key = &match->key;net/netfilter/nf_flow_table_offload.c line 65
Quoted 10 lines
net/netfilter/nf_flow_table_offload.c new file +10
-0+ +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_CONTROL , control); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_BASIC , basic); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_IPV4_ADDRS , ipv4); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_TCP , tcp); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_PORTS , tp); + +switch (tuple->l3proto) { +case AF_INET : + key->control.addr_type =FLOW_DISSECTOR_KEY_IPV4_ADDRS ;Is it intentional that
mask->control.addr_typenever gets set?
It should be set.
Quoted 2 lines
-ed
Pablo Neira Ayuso
maintainerRe: [PATCH net-next 6/6] netfilter: nf_flow_table: hardware offload support
Quoted 2 lines · Paul Blakey
On 19/03/2020 17:57, Edward Cree wrote:
[...]
Quoted 19 lines
Quoted 10 lines
<snip>
Quoted 9 lines
net/netfilter/nf_flow_table_offload.c new file +9
-0+static int nf_flow_rule_match (struct nf_flow_match *match, +const struct flow_offload_tuple *tuple) +{ +struct nf_flow_key *mask = &match->mask; +struct nf_flow_key *key = &match->key; + +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_CONTROL , control); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_BASIC , basic); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_IPV4_ADDRS , ipv4);Quoted 8 lines
net/netfilter/nf_flow_table_offload.c line 65
Quoted 7 lines
net/netfilter/nf_flow_table_offload.c new file +6
-0+NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_TCP , tcp); +NF_FLOW_DISSECTOR (match,FLOW_DISSECTOR_KEY_PORTS , tp); + +switch (tuple->l3proto) { +case AF_INET : + key->control.addr_type =FLOW_DISSECTOR_KEY_IPV4_ADDRS ;Is it intentional that
mask->control.addr_typenever gets set?
Quoted 1 line
It should be set.
I'd be glad to take a patch for this into nf.git
Thanks.
David Miller
maintainerFrom: Pablo Neira Ayuso <[email protected]>
Date: Tue, 12 Nov 2019 00:29:50 +0100
Quoted 4 lines
The following patchset adds hardware offload support for the flowtable infrastructure [1]. This infrastructure provides a fast datapath for the classic Linux forwarding path that users can enable through policy, eg.
...
Series applied, thanks.