the indirect flow_block offload, revisited
Subsystems (8)
Pablo Neira Ayuso
maintainercover the indirect flow_block offload, revisited
Hi,
This patchset fixes the indirect flow_block support for the tc CT action offload. Please, note that this batch is probably slightly large for the net tree, however, I could not find a simple incremental fix.
= The problem
The nf_flow_table_indr_block_cb() function provides the tunnel netdevice and the indirect flow_block driver callback. From this tunnel netdevice, it is not possible to obtain the tc CT flow_block. Note that tc qdisc and netfilter backtrack from the tunnel netdevice to the tc block / netfilter chain to reach the flow_block object. This allows them to clean up the hardware offload rules if the tunnel device is removed.
= What is the indirect flow_block infrastructure?
The indirect flow_block infrastructure allows drivers to offload tc/netfilter rules that belong to software tunnel netdevices, e.g. vxlan.
This indirect flow_block infrastructure relates tunnel netdevices with drivers because there is no obvious way to relate these two things from the control plane.
= How does the indirect flow_block work before this patchset?
Front-ends register the indirect flow_block callback through flow_indr_add_block_cb() if they support for offloading tunnel netdevices.
== Setting up an indirect flow_block
1) Drivers track tunnel netdevices via NETDEV_{REGISTER,UNREGISTER} events. If there is a new tunnel netdevice that the driver can offload, then the driver invokes __flow_indr_block_cb_register() with the new tunnel netdevice and the driver callback. The __flow_indr_block_cb_register() call iterates over the list of the front-end callbacks.
2) The front-end callback sets up the flow_block_offload structure and it invokes the driver callback to set up the flow_block.
3) The driver callback now registers the flow_block structure and it returns the flow_block back to the front-end.
4) The front-end gets the flow_block object and it is now ready to offload rules for this tunnel netdevice.
A simplified callgraph is represented below.
Front-end Driver
NETDEV_REGISTER
|
__flow_indr_block_cb_register(netdev, cb_priv, driver_cb)
| [1]
.---------- frontend_indr_block_cb(cb_priv, driver_cb)
|
setup_flow_block_offload(bo)
| [2]
driver_cb(bo, cb_priv) ---------------.
|
set up flow_blocks [3]
| add rules to flow_block <-------------'
TC_SETUP_CLSFLOWER [4]
== Releasing the indirect flow_block
There are two possibilities, either tunnel netdevice is removed or a netdevice (port representor) is removed.
=== Tunnel netdevice is removed
Driver waits for the NETDEV_UNREGISTER event that announces the tunnel netdevice removal. Then, it calls __flow_indr_block_cb_unregister() to remove the flow_block and rules. Callgraph is very similar to the one described above.
=== Netdevice is removed (port representor)
Driver calls __flow_indr_block_cb_unregister() to remove the existing netfilter/tc rule that belong to the tunnel netdevice.
= How does the indirect flow_block work after this patchset?
Drivers register the indirect flow_block setup callback through flow_indr_dev_register() if they support for offloading tunnel netdevices.
== Setting up an indirect flow_block
1) Frontends check if dev->netdev_ops->ndo_setup_tc is unset. If so, frontends call flow_indr_dev_setup_offload(). This call invokes the drivers' indirect flow_block setup callback.
2) The indirect flow_block setup callback sets up a flow_block structure which relates the tunnel netdevice and the driver.
3) The front-end uses flow_block and offload the rules.
Note that the operational to set up (non-indirect) flow_block is very similar.
== Releasing the indirect flow_block
=== Tunnel netdevice is removed
This calls flow_indr_dev_setup_offload() to set down the flow_block and remove the offloaded rules. This alternate path is exercised if dev->netdev_ops->ndo_setup_tc is unset.
=== Netdevice is removed (port representor)
If a netdevice is removed, then it might need to to clean up the offloaded tc/netfilter rules that belongs to the tunnel netdevice:
1) The driver invokes flow_indr_dev_unregister() when a netdevice is removed.
2) This call iterates over the existing indirect flow_blocks and it invokes the cleanup callback to let the front-end remove the tc/netfilter rules. The cleanup callback already provides the flow_block that the front-end needs to clean up.
Front-end Driver
|
flow_indr_dev_unregister(...)
|
iterate over list of indirect flow_block
and invoke cleanup callback
|
.-----------------------------
|
.
frontend_flow_block_cleanup(flow_block)
.
|
\/ remove rules to flow_block
TC_SETUP_CLSFLOWER
= About this patchset
This patchset aims to address the existing TC CT problem while simplifying the indirect flow_block infrastructure. Saving 300 LoC in the flow_offload core and the drivers. The operational gets aligned with the (non-indirect) flow_blocks logic. Patchset is composed of:
Patch #1 add nf_flow_table_gc_cleanup() which is required by the
netfilter's flowtable new indirect flow_block approach.
Patch #2 adds the flow_block_indr object which is actually part of
of the flow_block object. This stores the indirect flow_block metadata such as the tunnel netdevice owner and the cleanup callback (in case the tunnel netdevice goes away).
This patch adds flow_indr_dev_{un}register() to allow drivers to offer netdevice tunnel hardware offload to the front-ends. Then, front-ends call flow_indr_dev_setup_offload() to invoke the drivers to set up the (indirect) flow_block.
Patch #3 add the tcf_block_offload_init() helper function, this is
a preparation patch to adapt the tc front-end to use this new indirect flow_block infrastructure.
Patch #4 updates the tc and netfilter front-ends to use the new
indirect flow_block infrastructure.
Patch #5 updates the mlx5 driver to use the new indirect flow_block
infrastructure.
Patch #6 updates the nfp driver to use the new indirect flow_block
infrastructure.
Patch #7 updates the bnxt driver to use the new indirect flow_block
infrastructure.
Patch #8 removes the indirect flow_block infrastructure version 1,
now that frontends and drivers have been translated to version 2 (coming in this patchset).
Please, apply.
Pablo Neira Ayuso (8):
netfilter: nf_flowtable: expose nf_flow_table_gc_cleanup()
net: flow_offload: consolidate indirect flow_block infrastructure
net: cls_api: add tcf_block_offload_init()
net: use flow_indr_dev_setup_offload()
mlx5: update indirect block support
nfp: update indirect block support
bnxt_tc: update indirect block support
net: remove indirect block netdev event registration
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 - drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c | 51 +-- .../net/ethernet/mellanox/mlx5/core/en_rep.c | 83 +---- .../net/ethernet/mellanox/mlx5/core/en_rep.h | 5 - .../net/ethernet/netronome/nfp/flower/main.c | 11 +- .../net/ethernet/netronome/nfp/flower/main.h | 7 +- .../ethernet/netronome/nfp/flower/offload.c | 35 +- include/net/flow_offload.h | 28 +- include/net/netfilter/nf_flow_table.h | 2 + net/core/flow_offload.c | 301 +++++++----------- net/netfilter/nf_flow_table_core.c | 6 +- net/netfilter/nf_flow_table_offload.c | 85 +---- net/netfilter/nf_tables_offload.c | 69 ++-- net/sched/cls_api.c | 157 +++------ 14 files changed, 251 insertions(+), 590 deletions(-)
--
2.20.1
Pablo Neira Ayuso
maintainerPATCH 1/8 netfilter: nf_flowtable: expose nf_flow_table_gc_cleanup()
This function schedules the flow teardown state and it forces a gc run.
--- include/net/netfilter/nf_flow_table.h | 2 ++ net/netfilter/nf_flow_table_core.c | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-)
include/net/netfilter/nf_flow_table.h +2 -0
@@ -174,6 +174,8 @@ void flow_offload_refresh(struct nf_flowtable *flow_table,struct flow_offload_tuple_rhash *flow_offload_lookup (struct nf_flowtable *flow_table,struct flow_offload_tuple *tuple); +void nf_flow_table_gc_cleanup (struct nf_flowtable *flowtable, +struct net_device *dev);void nf_flow_table_cleanup (struct net_device *dev);int nf_flow_table_init (struct nf_flowtable *flow_table);
net/netfilter/nf_flow_table_core.c +3 -3
@@ -586,8 +586,8 @@ static void nf_flow_table_do_cleanup(struct flow_offload *flow, void *data)flow_offload_teardown (flow); } -static void nf_flow_table_iterate_cleanup (struct nf_flowtable *flowtable, -struct net_device *dev) +void nf_flow_table_gc_cleanup (struct nf_flowtable *flowtable, +struct net_device *dev) {nf_flow_table_iterate (flowtable, nf_flow_table_do_cleanup, dev);flush_delayed_work (&flowtable->gc_work);
@@ -600,7 +600,7 @@ void nf_flow_table_cleanup(struct net_device *dev)mutex_lock (&flowtable_lock);list_for_each_entry (flowtable, &flowtables, list) -nf_flow_table_iterate_cleanup (flowtable, dev); +nf_flow_table_gc_cleanup (flowtable, dev);mutex_unlock (&flowtable_lock); }EXPORT_SYMBOL_GPL (nf_flow_table_cleanup);
Signature · 1 line
2.20.1
Pablo Neira Ayuso
maintainerPATCH 2/8 net: flow_offload: consolidate indirect flow_block infrastructure
In reply to Pablo Neira AyusoTunnel devices provide no dev->netdev_ops->ndo_setup_tc(...) interface. The tunnel device and route control plane does not provide an obvious way to relate tunnel and physical devices.
This patch allows drivers to register a tunnel device offload handler for the tc and netfilter frontends through flow_indr_dev_register() and flow_indr_dev_unregister().
The frontend calls flow_indr_dev_setup_offload() that iterates over the list of drivers that are offering tunnel device hardware offload support and it sets up the flow block for this tunnel device.
If the driver module is removed, the indirect flow_block ends up with a stale callback reference. The module removal path triggers the dev_shutdown() path to remove the qdisc and the flow_blocks for the physical devices. However, this is not useful for tunnel devices, where relation between the physical and the tunnel device is not explicit.
This patch introduces a cleanup callback that is invoked when the driver module is removed to clean up the tunnel device flow_block. This patch defines struct flow_block_indr and it uses it from flow_block_cb to store the information that front-end requires to perform the flow_block_cb cleanup on module removal.
--- include/net/flow_offload.h | 19 +++++ net/core/flow_offload.c | 157 +++++++++++++++++++++++++++++++++++++ 2 files changed, 176 insertions(+)
include/net/flow_offload.h +19 -0
@@ -430,6 +430,16 @@ enum tc_setup_type;typedef int flow_setup_cb_t (enum tc_setup_type type,void *type_data,void *cb_priv); +struct flow_block_cb ; + +struct flow_block_indr { +struct list_head list; +struct net_device *dev; +enum flow_block_binder_type binder_type; +void *data; +void (*cleanup)(struct flow_block_cb *block_cb); +}; +struct flow_block_cb {struct list_head driver_list;struct list_head list;
@@ -437,6 +447,7 @@ struct flow_block_cb {void *cb_ident;void *cb_priv;void (*release)(void *cb_priv); +struct flow_block_indr indr;unsigned int refcnt; };
@@ -510,6 +521,14 @@ static inline void flow_block_init(struct flow_block *flow_block)typedef int flow_indr_block_bind_cb_t (struct net_device *dev,void *cb_priv,enum tc_setup_type type,void *type_data); +int flow_indr_dev_register (flow_indr_block_bind_cb_t *cb,void *cb_priv); +void flow_indr_dev_unregister (flow_indr_block_bind_cb_t *cb,void *cb_priv, +flow_setup_cb_t *setup_cb); +int flow_indr_dev_setup_offload (struct net_device *dev, +enum tc_setup_type type,void *data, +struct flow_block_offload *bo, +void (*cleanup)(struct flow_block_cb *block_cb)); +typedef void flow_indr_block_cmd_t (struct net_device *dev,flow_indr_block_bind_cb_t *cb,void *cb_priv,enum flow_block_command command);
net/core/flow_offload.c +157 -0
163 lines not on this page · raw message
Signature · 1 line
2.20.1
Pablo Neira Ayuso
maintainerPATCH 3/8 net: cls_api: add tcf_block_offload_init()
In reply to Pablo Neira AyusoAdd a helper function to initialize the flow_block_offload structure.
--- net/sched/cls_api.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-)
net/sched/cls_api.c +19 -7
@@ -692,6 +692,22 @@ static void tc_indr_block_get_and_cmd(struct net_device *dev,tc_indr_block_cmd (dev, block, cb, cb_priv, command, false); } +static void tcf_block_offload_init (struct flow_block_offload *bo, +struct net_device *dev, +enum flow_block_command command, +enum flow_block_binder_type binder_type, +struct flow_block *flow_block, +bool shared,struct netlink_ext_ack *extack) +{ + bo->net =dev_net (dev); + bo->command = command; + bo->binder_type = binder_type; + bo->block = flow_block; + bo->block_shared = shared; + bo->extack = extack; +INIT_LIST_HEAD (&bo->cb_list); +} + static void tc_indr_block_call(struct tcf_block *block, struct net_device *dev, struct tcf_block_ext_info *ei,
@@ -726,13 +742,9 @@ static int tcf_block_offload_cmd(struct tcf_block *block,struct flow_block_offload bo = {};int err; - bo.net =dev_net (dev); - bo.command = command; - bo.binder_type = ei->binder_type; - bo.block = &block->flow_block; - bo.block_shared =tcf_block_shared (block); - bo.extack = extack; -INIT_LIST_HEAD (&bo.cb_list); +tcf_block_offload_init (&bo, dev, command, ei->binder_type, + &block->flow_block,tcf_block_shared (block), + extack); err = dev->netdev_ops->ndo_setup_tc (dev,TC_SETUP_BLOCK , &bo);if (err <0 )
Signature · 1 line
2.20.1
Pablo Neira Ayuso
maintainerPATCH 4/8 net: use flow_indr_dev_setup_offload()
In reply to Pablo Neira AyusoUpdate existing frontends to use flow_indr_dev_setup_offload().
This new function must be called if ->ndo_setup_tc is unset to deal with tunnel devices.
If there is no driver that is subscribed to new tunnel device flow_block bindings, then this function bails out with EOPNOTSUPP.
If the driver module is removed, the ->cleanup() callback removes the entries that belong to this tunnel device. This cleanup procedures is triggered when the device unregisters the tunnel device offload handler.
--- net/netfilter/nf_flow_table_offload.c | 19 ++++++--- net/netfilter/nf_tables_offload.c | 28 +++++++++++-- net/sched/cls_api.c | 58 +++++++++++++-------------- 3 files changed, 67 insertions(+), 38 deletions(-)
net/netfilter/nf_flow_table_offload.c +14 -5
@@ -933,6 +933,18 @@ static void nf_flow_table_block_offload_init(struct flow_block_offload *bo,INIT_LIST_HEAD (&bo->cb_list); } +static void nf_flow_table_indr_cleanup (struct flow_block_cb *block_cb) +{ +struct nf_flowtable *flowtable = block_cb->indr.data; +struct net_device *dev = block_cb->indr.dev; + +nf_flow_table_gc_cleanup (flowtable, dev); +down_write (&flowtable->flow_block_lock); +list_del (&block_cb->list); +flow_block_cb_free (block_cb); +up_write (&flowtable->flow_block_lock); +} + static int nf_flow_table_indr_offload_cmd(struct flow_block_offload *bo, struct nf_flowtable *flowtable, struct net_device *dev,
@@ -941,12 +953,9 @@ static int nf_flow_table_indr_offload_cmd(struct flow_block_offload *bo, {nf_flow_table_block_offload_init (bo,dev_net (dev), cmd, flowtable, extack); -flow_indr_block_call (dev, bo, cmd,TC_SETUP_FT ); -if (list_empty (&bo->cb_list)) -return -EOPNOTSUPP ; - -return 0 ; +return flow_indr_dev_setup_offload (dev,TC_SETUP_FT , flowtable, bo, + nf_flow_table_indr_cleanup); } static int nf_flow_table_offload_cmd(struct flow_block_offload *bo,
net/netfilter/nf_tables_offload.c +24 -4
@@ -304,21 +304,41 @@ static void nft_indr_block_ing_cmd(struct net_device *dev,nft_block_setup (chain, &bo, cmd); } -static int nft_indr_block_offload_cmd (struct nft_base_chain *chain, +static void nft_indr_block_cleanup (struct flow_block_cb *block_cb) +{ +struct nft_base_chain *basechain = block_cb->indr.data; +struct net_device *dev = block_cb->indr.dev; +struct netlink_ext_ack extack = {}; +struct net *net =dev_net (dev); +struct flow_block_offload bo; + +nft_flow_block_offload_init (&bo,dev_net (dev),FLOW_BLOCK_UNBIND , + basechain, &extack); +mutex_lock (&net->nft.commit_mutex); +list_move (&block_cb->list, &bo.cb_list); +nft_flow_offload_unbind (&bo, basechain); +mutex_unlock (&net->nft.commit_mutex); +} + +static int nft_indr_block_offload_cmd (struct nft_base_chain *basechain,struct net_device *dev,enum flow_block_command cmd) {struct netlink_ext_ack extack = {};struct flow_block_offload bo; +int err; -nft_flow_block_offload_init (&bo,dev_net (dev), cmd, chain, &extack); +nft_flow_block_offload_init (&bo,dev_net (dev), cmd, basechain, &extack); -flow_indr_block_call (dev, &bo, cmd,TC_SETUP_BLOCK ); + err =flow_indr_dev_setup_offload (dev,TC_SETUP_BLOCK , basechain, &bo, + nft_indr_block_cleanup); +if (err <0 ) +return err;if (list_empty (&bo.cb_list))return -EOPNOTSUPP ; -return nft_block_setup (chain, &bo, cmd); +return nft_block_setup (basechain, &bo, cmd); }#define FLOW_SETUP_BLOCK TC_SETUP_BLOCK
net/sched/cls_api.c +29 -29
101 lines not on this page · raw message
Signature · 1 line
2.20.1
Pablo Neira Ayuso
maintainerPATCH 5/8 mlx5: update indirect block support
In reply to Pablo Neira AyusoRegister ndo callback via flow_indr_dev_register() and flow_indr_dev_unregister().
No need for mlx5e_rep_indr_clean_block_privs() since flow_block_cb_free() already releases the internal mapping via ->release callback, which in this case is mlx5e_rep_indr_tc_block_unbind().
--- .../net/ethernet/mellanox/mlx5/core/en_rep.c | 83 +++---------------- .../net/ethernet/mellanox/mlx5/core/en_rep.h | 5 -- 2 files changed, 10 insertions(+), 78 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +10 -73
@@ -65,9 +65,6 @@ struct mlx5e_rep_indr_block_priv {struct list_head list; }; -static void mlx5e_rep_indr_unregister_block (struct mlx5e_rep_priv *rpriv, -struct net_device *netdev); -static void mlx5e_rep_get_drvinfo (struct net_device *dev,struct ethtool_drvinfo *drvinfo) {
@@ -680,17 +677,6 @@ mlx5e_rep_indr_block_priv_lookup(struct mlx5e_rep_priv *rpriv,return NULL ; } -static void mlx5e_rep_indr_clean_block_privs (struct mlx5e_rep_priv *rpriv) -{ -struct mlx5e_rep_indr_block_priv *cb_priv, *temp; -struct list_head *head = &rpriv->uplink_priv.tc_indr_block_priv_list; - -list_for_each_entry_safe (cb_priv, temp, head, list) { -mlx5e_rep_indr_unregister_block (rpriv, cb_priv->netdev); -kfree (cb_priv); - } -} - static int mlx5e_rep_indr_offload(struct net_device *netdev, struct flow_cls_offload *flower,
@@ -794,9 +780,14 @@ mlx5e_rep_indr_setup_block(struct net_device *netdev,struct flow_block_offload *f, flow_setup_cb_t *setup_cb) { + struct mlx5e_priv *priv = netdev_priv(rpriv->netdev);struct mlx5e_rep_indr_block_priv *indr_priv;struct flow_block_cb *block_cb; +if (!mlx5e_tc_tun_device_to_offload (priv, netdev) && + !(is_vlan_dev (netdev) &&vlan_dev_real_dev (netdev) == rpriv->netdev)) +return -EOPNOTSUPP ; +if (f->binder_type !=FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS )return -EOPNOTSUPP ;
@@ -863,53 +854,6 @@ int mlx5e_rep_indr_setup_cb(struct net_device *netdev, void *cb_priv, } } -static int mlx5e_rep_indr_register_block (struct mlx5e_rep_priv *rpriv, -struct net_device *netdev) -{ -int err; - - err =__flow_indr_block_cb_register (netdev, rpriv, - mlx5e_rep_indr_setup_cb, - rpriv); -if (err) { -struct mlx5e_priv *priv =netdev_priv (rpriv->netdev); - -mlx5_core_err (priv->mdev,"Failed to register remote block notifier for %s err=%d\n" , -netdev_name (netdev), err); - } -return err; -} - -static void mlx5e_rep_indr_unregister_block (struct mlx5e_rep_priv *rpriv, -struct net_device *netdev) -{ -__flow_indr_block_cb_unregister (netdev, mlx5e_rep_indr_setup_cb, - rpriv); -} - -static int mlx5e_nic_rep_netdevice_event (struct notifier_block *nb, -unsigned long event,void *ptr) -{ - struct mlx5e_rep_priv *rpriv = container_of(nb, struct mlx5e_rep_priv, - uplink_priv.netdevice_nb); -struct mlx5e_priv *priv =netdev_priv (rpriv->netdev); -struct net_device *netdev =netdev_notifier_info_to_dev (ptr); - -if (!mlx5e_tc_tun_device_to_offload (priv, netdev) && - !(is_vlan_dev (netdev) &&vlan_dev_real_dev (netdev) == rpriv->netdev)) -return NOTIFY_OK ; - -switch (event) { -case NETDEV_REGISTER : -mlx5e_rep_indr_register_block (rpriv, netdev); -break ; -case NETDEV_UNREGISTER : -mlx5e_rep_indr_unregister_block (rpriv, netdev); -break ; - } -return NOTIFY_OK ; -} - static void mlx5e_rep_queue_neigh_update_work(struct mlx5e_priv *priv, struct mlx5e_neigh_hash_entry *nhe,
@@ -1806,12 +1750,10 @@ static int mlx5e_init_uplink_rep_tx(struct mlx5e_rep_priv *rpriv)/* init indirect block notifications */ INIT_LIST_HEAD (&uplink_priv->tc_indr_block_priv_list); - uplink_priv->netdevice_nb.notifier_call = mlx5e_nic_rep_netdevice_event; - err =register_netdevice_notifier_dev_net (rpriv->netdev, - &uplink_priv->netdevice_nb, - &uplink_priv->netdevice_nn); + + err =flow_indr_dev_register (mlx5e_rep_indr_setup_cb, rpriv);if (err) { -mlx5_core_err (priv->mdev,"Failed to register netdev notifier\n" ); +mlx5_core_err (priv->mdev,"Failed to register indirect block notifier\n" ); goto tc_esw_cleanup; }
@@ -1848,13 +1790,8 @@ static int mlx5e_init_rep_tx(struct mlx5e_priv *priv)static void mlx5e_cleanup_uplink_rep_tx (struct mlx5e_rep_priv *rpriv) { -struct mlx5_rep_uplink_priv *uplink_priv = &rpriv->uplink_priv; - -/* clean indirect TC block notifications */ -unregister_netdevice_notifier_dev_net (rpriv->netdev, - &uplink_priv->netdevice_nb, - &uplink_priv->netdevice_nn); -mlx5e_rep_indr_clean_block_privs (rpriv); +flow_indr_dev_unregister (mlx5e_rep_indr_setup_cb, rpriv, + mlx5e_rep_indr_setup_tc_cb);/* delete shared tc flow table */ mlx5e_tc_esw_cleanup (&rpriv->uplink_priv.tc_ht);
drivers/net/ethernet/mellanox/mlx5/core/en_rep.h +0 -5
@@ -68,13 +68,8 @@ struct mlx5_rep_uplink_priv {* tc_indr_block_cb_priv_list is used to lookup indirect callback * private data * -* netdevice_nb is the netdev events notifier - used to register -* tunnel devices for block events -* */ struct list_head tc_indr_block_priv_list; -struct notifier_block netdevice_nb; -struct netdev_net_notifier netdevice_nn;struct mlx5_tun_entropy tun_entropy;
Signature · 1 line
2.20.1
Pablo Neira Ayuso
maintainerPATCH 6/8 nfp: update indirect block support
In reply to Pablo Neira AyusoRegister ndo callback via flow_indr_dev_register() and flow_indr_dev_unregister().
--- .../net/ethernet/netronome/nfp/flower/main.c | 11 +++--- .../net/ethernet/netronome/nfp/flower/main.h | 7 ++-- .../ethernet/netronome/nfp/flower/offload.c | 35 ++++--------------- 3 files changed, 17 insertions(+), 36 deletions(-)
drivers/net/ethernet/netronome/nfp/flower/main.c +7 -4
@@ -783,6 +783,10 @@ static int nfp_flower_init(struct nfp_app *app)nfp_warn (app->cpp,"Flow mod/merge not supported by FW.\n" ); } + err =flow_indr_dev_register (nfp_flower_indr_setup_tc_cb, app); +if (err) + goto err_lag_clean; +if (app_priv->flower_ext_feats &NFP_FL_FEATS_VF_RLIM )nfp_flower_qos_init (app);
@@ -810,6 +814,9 @@ static void nfp_flower_clean(struct nfp_app *app)skb_queue_purge (&app_priv->cmsg_skbs_low);flush_work (&app_priv->cmsg_work); +flow_indr_dev_unregister (nfp_flower_indr_setup_tc_cb, app, + nfp_flower_setup_indr_block_cb); +if (app_priv->flower_ext_feats &NFP_FL_FEATS_VF_RLIM )nfp_flower_qos_cleanup (app);
@@ -913,10 +920,6 @@ nfp_flower_netdev_event(struct nfp_app *app, struct net_device *netdev,return ret; } - ret =nfp_flower_reg_indir_block_handler (app, netdev, event); -if (ret &NOTIFY_STOP_MASK ) -return ret; - ret =nfp_flower_internal_port_event_handler (app, netdev, event);if (ret &NOTIFY_STOP_MASK )return ret;
drivers/net/ethernet/netronome/nfp/flower/main.h +4 -3
@@ -444,9 +444,10 @@ void nfp_flower_qos_cleanup(struct nfp_app *app);int nfp_flower_setup_qos_offload (struct nfp_app *app,struct net_device *netdev,struct tc_cls_matchall_offload *flow);void nfp_flower_stats_rlim_reply (struct nfp_app *app,struct sk_buff *skb); -int nfp_flower_reg_indir_block_handler (struct nfp_app *app, -struct net_device *netdev, -unsigned long event); +int nfp_flower_indr_setup_tc_cb (struct net_device *netdev,void *cb_priv, +enum tc_setup_type type,void *type_data); +int nfp_flower_setup_indr_block_cb (enum tc_setup_type type,void *type_data, +void *cb_priv);void __nfp_flower_non_repr_priv_get (struct nfp_flower_non_repr_priv *non_repr_priv);
drivers/net/ethernet/netronome/nfp/flower/offload.c +6 -29
@@ -1618,8 +1618,8 @@ nfp_flower_indr_block_cb_priv_lookup(struct nfp_app *app,return NULL ; } -static int nfp_flower_setup_indr_block_cb (enum tc_setup_type type, -void *type_data,void *cb_priv) +int nfp_flower_setup_indr_block_cb (enum tc_setup_type type, +void *type_data,void *cb_priv) {struct nfp_flower_indr_block_cb_priv *priv = cb_priv;struct flow_cls_offload *flower = type_data;
@@ -1707,10 +1707,13 @@ nfp_flower_setup_indr_tc_block(struct net_device *netdev, struct nfp_app *app,return 0 ; } -static int +int nfp_flower_indr_setup_tc_cb (struct net_device *netdev,void *cb_priv,enum tc_setup_type type,void *type_data) { +if (!nfp_fl_is_netdev_to_offload (netdev)) +return -EOPNOTSUPP ; +switch (type) {case TC_SETUP_BLOCK : return nfp_flower_setup_indr_tc_block(netdev, cb_priv,
@@ -1719,29 +1722,3 @@ nfp_flower_indr_setup_tc_cb(struct net_device *netdev, void *cb_priv,return -EOPNOTSUPP ; } } - -int nfp_flower_reg_indir_block_handler (struct nfp_app *app, -struct net_device *netdev, -unsigned long event) -{ -int err; - -if (!nfp_fl_is_netdev_to_offload (netdev)) -return NOTIFY_OK ; - -if (event ==NETDEV_REGISTER ) { - err =__flow_indr_block_cb_register (netdev, app, - nfp_flower_indr_setup_tc_cb, - app); -if (err) -nfp_flower_cmsg_warn (app, -"Indirect block reg failed - %s\n" , - netdev->name); - }else if (event ==NETDEV_UNREGISTER ) { -__flow_indr_block_cb_unregister (netdev, - nfp_flower_indr_setup_tc_cb, - app); - } - -return NOTIFY_OK ; -}
Signature · 1 line
2.20.1
Pablo Neira Ayuso
maintainerPATCH 7/8 bnxt_tc: update indirect block support
In reply to Pablo Neira AyusoRegister ndo callback via flow_indr_dev_register() and flow_indr_dev_unregister().
--- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 - drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c | 51 +++++--------------- 2 files changed, 12 insertions(+), 40 deletions(-)
drivers/net/ethernet/broadcom/bnxt/bnxt.h +0 -1
@@ -1857,7 +1857,6 @@ struct bnxt {u8 dsn[8 ];struct bnxt_tc_info *tc_info;struct list_head tc_indr_block_list; -struct notifier_block tc_netdev_nb;struct dentry *debugfs_pdev;struct device *hwmon_dev; };
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +12 -39
80 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 7/8 net] bnxt_tc: update indirect block support
Quoted 100 lines · Pablo Neira Ayuso
Register ndo callback via
flow_indr_dev_register()andflow_indr_dev_unregister().Signed-off-by Pablo Neira Ayuso <[email protected]>--- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 - drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c | 51 +++++--------------- 2 files changed, 12 insertions(+), 40 deletions(-)drivers/net/ethernet/broadcom/bnxt/bnxt.h +0
-1@@ -1857,7 +1857,6 @@ struct bnxt {u8 dsn[8 ];struct bnxt_tc_info *tc_info;struct list_head tc_indr_block_list; -struct notifier_block tc_netdev_nb;struct dentry *debugfs_pdev;struct device *hwmon_dev; };drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +10
-38@@ -1939,53 +1939,25 @@ static int bnxt_tc_setup_indr_block(struct net_device *netdev, struct bnxt *bp,return 0 ; } -static int bnxt_tc_setup_indr_cb (struct net_device *netdev,void *cb_priv, -enum tc_setup_type type,void *type_data) -{ -switch (type) { -case TC_SETUP_BLOCK : -return bnxt_tc_setup_indr_block (netdev, cb_priv, type_data); -default : -return -EOPNOTSUPP ; - } -} -static bool bnxt_is_netdev_indr_offload (struct net_device *netdev) {return netif_is_vxlan (netdev); } -static int bnxt_tc_indr_block_event (struct notifier_block *nb, -unsigned long event,void *ptr) +static int bnxt_tc_setup_indr_cb (struct net_device *netdev,void *cb_priv, +enum tc_setup_type type,void *type_data) { -struct net_device *netdev; -struct bnxt *bp; -int rc; - - netdev =netdev_notifier_info_to_dev (ptr);if (!bnxt_is_netdev_indr_offload (netdev)) -return NOTIFY_OK ; - - bp = container_of(nb, struct bnxt, tc_netdev_nb); +return -EOPNOTSUPP ; -switch (event) { -case NETDEV_REGISTER : - rc =__flow_indr_block_cb_register (netdev, bp, - bnxt_tc_setup_indr_cb, - bp); -if (rc) -netdev_info (bp->dev, -"Failed to register indirect blk: dev: %s\n" , - netdev->name); -break ; -case NETDEV_UNREGISTER : -__flow_indr_block_cb_unregister (netdev, - bnxt_tc_setup_indr_cb, - bp); +switch (type) { +case TC_SETUP_BLOCK : +return bnxt_tc_setup_indr_block (netdev, cb_priv, type_data); +default :break ; } -return NOTIFY_DONE ; +return -EOPNOTSUPP ; }static const struct rhashtable_params bnxt_tc_flow_ht_params = {@@ -2074,8 +2046,8 @@ int bnxt_init_tc(struct bnxt *bp)/* init indirect block notifications */ INIT_LIST_HEAD (&bp->tc_indr_block_list); - bp->tc_netdev_nb.notifier_call = bnxt_tc_indr_block_event; - rc =register_netdevice_notifier (&bp->tc_netdev_nb); + + rc =flow_indr_dev_register (bnxt_tc_setup_indr_cb, bp);if (!rc)return 0 ;
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c void bnxt_shutdown_tc(struct bnxt *bp) · line 2077
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +2
-1@@ -2101,7 +2073,8 @@ void bnxt_shutdown_tc(struct bnxt *bp)if (!bnxt_tc_flower_enabled (bp))return ; -unregister_netdevice_notifier (&bp->tc_netdev_nb); +flow_indr_dev_unregister (bnxt_tc_setup_indr_cb, bp, + bnxt_tc_setup_indr_block_cb);
Why does the driver need to provide the "cb" again during unregister, since both "cb" and "cb_priv" are already provided during register() ? This interface could be simplified/improved if flow_indr_dev_register() returns an opaque handle to the object it creates (struct flow_indr_dev *) ? The driver should just pass this handle during unregistration. Also, why do we need the extra (3rd) argument (flow_setup_cb_t / bnxt_tc_setup_indr_block_cb) during unreg ? It is handled internally by the driver as a part of FLOW_BLOCK_BIND / UNBIND ?
Thanks,
-Harsha
Quoted 6 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +0
-1rhashtable_destroy (&tc_info->flow_table);rhashtable_destroy (&tc_info->l2_table);rhashtable_destroy (&tc_info->decap_l2_table); --2.20.1
Pablo Neira Ayuso
maintainerRe: [PATCH 7/8 net] bnxt_tc: update indirect block support
Hi,
I overlook this email, sorry.
On Tue, May 19, 2020 at 02:23:54PM +0530, Sriharsha Basavapatna wrote:
[...]
Quoted 4 lines
Quoted 4 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +0
-0@@ -2101,7 +2073,8 @@ void bnxt_shutdown_tc(struct bnxt *bp)if (!bnxt_tc_flower_enabled (bp))return ;
Quoted 9 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c void bnxt_shutdown_tc(struct bnxt *bp) · line 2077
Quoted 3 lines
drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +2
-1-unregister_netdevice_notifier (&bp->tc_netdev_nb); +flow_indr_dev_unregister (bnxt_tc_setup_indr_cb, bp, + bnxt_tc_setup_indr_block_cb);Why does the driver need to provide the "cb" again during unregister, since both "cb" and "
cb_priv" are already provided duringregister()? This interface could be simplified/improved ifflow_indr_dev_register()returns an opaque handle to the object it creates (structflow_indr_dev*) ?
Probably, at the expense to storing this in the netdev private area.
Quoted 5 lines
The driver should just pass this
handle during unregistration. Also, why do we need the extra (3rd) argument (flow_setup_cb_t/bnxt_tc_setup_indr_block_cb) during unreg ? It is handled internally by the driver as a part ofFLOW_BLOCK_BIND/ UNBIND ?
flow_indr_dev_unregister() needs bnxt_tc_setup_indr_block_cb to identify what indirect flow_blocks need to be cleaned up before this representor is gone.
Pablo Neira Ayuso
maintainerPATCH 8/8 net: remove indirect block netdev event registration
In reply to Pablo Neira AyusoDrivers do not register to netdev events to set up indirect blocks anymore. Remove __flow_indr_block_cb_register() and __flow_indr_block_cb_unregister().
The frontends set up the callbacks through flow_indr_dev_setup_block()
--- include/net/flow_offload.h | 9 - net/core/flow_offload.c | 238 -------------------------- net/netfilter/nf_flow_table_offload.c | 66 ------- net/netfilter/nf_tables_offload.c | 53 +----- net/sched/cls_api.c | 79 --------- 5 files changed, 1 insertion(+), 444 deletions(-)
include/net/flow_offload.h +0 -9
@@ -533,15 +533,6 @@ typedef void flow_indr_block_cmd_t(struct net_device *dev, flow_indr_block_bind_cb_t *cb, void *cb_priv, enum flow_block_command command); -struct flow_indr_block_entry { -flow_indr_block_cmd_t *cb; -struct list_head list; -}; - -void flow_indr_add_block_cb (struct flow_indr_block_entry *entry); - -void flow_indr_del_block_cb (struct flow_indr_block_entry *entry); -int __flow_indr_block_cb_register (struct net_device *dev,void *cb_priv,flow_indr_block_bind_cb_t *cb,void *cb_ident);
net/core/flow_offload.c +0 -238
241 lines not on this page · raw message
net/netfilter/nf_flow_table_offload.c +0 -66
82 lines not on this page · raw message
net/netfilter/nf_tables_offload.c +1 -52
80 lines not on this page · raw message
net/sched/cls_api.c +0 -79
97 lines not on this page · raw message
Signature · 1 line
2.20.1
Re: [PATCH 8/8 net] net: remove indirect block netdev event registration
Quoted 7 lines · Pablo Neira Ayuso
Drivers do not register to netdev events to set up indirect blocks anymore. Remove
__flow_indr_block_cb_register()and__flow_indr_block_cb_unregister().The frontends set up the callbacks through
flow_indr_dev_setup_block()Signed-off-by Pablo Neira Ayuso <[email protected]>
This commit failed to remove the prototypes from the header file:include/net/flow_offload.h
Thanks,
Jake
Pablo Neira Ayuso
maintainerRe: [PATCH 8/8 net] net: remove indirect block netdev event registration
Quoted 7 lines · Jacob Keller
Quoted 4 lines · Pablo Neira Ayuso
Drivers do not register to netdev events to set up indirect blocks anymore. Remove
__flow_indr_block_cb_register()and__flow_indr_block_cb_unregister().
Quoted 6 lines
Quoted 3 lines
The frontends set up the callbacks through
flow_indr_dev_setup_block()Signed-off-by Pablo Neira Ayuso <[email protected]>This commit failed to remove the prototypes from the header file:
include/net/flow_offload.h
Thanks for reporting.
I'm attaching a sketch, I will submit this formally later.
include/net/flow_offload.h +0 -24
@@ -542,28 +542,4 @@ int flow_indr_dev_setup_offload(struct net_device *dev,struct flow_block_offload *bo, void (*cleanup)(struct flow_block_cb *block_cb)); -typedef void flow_indr_block_cmd_t (struct net_device *dev, -flow_indr_block_bind_cb_t *cb,void *cb_priv, -enum flow_block_command command); - -int __flow_indr_block_cb_register (struct net_device *dev,void *cb_priv, -flow_indr_block_bind_cb_t *cb, -void *cb_ident); - -void __flow_indr_block_cb_unregister (struct net_device *dev, -flow_indr_block_bind_cb_t *cb, -void *cb_ident); - -int flow_indr_block_cb_register (struct net_device *dev,void *cb_priv, -flow_indr_block_bind_cb_t *cb,void *cb_ident); - -void flow_indr_block_cb_unregister (struct net_device *dev, -flow_indr_block_bind_cb_t *cb, -void *cb_ident); - -void flow_indr_block_call (struct net_device *dev, -struct flow_block_offload *bo, -enum flow_block_command command, -enum tc_setup_type type); -#endif /* _NET_FLOW_OFFLOAD_H */
Re: [PATCH 8/8 net] net: remove indirect block netdev event registration
Quoted 10 lines · Pablo Neira Ayuso
Quoted 9 lines · Jacob Keller
Quoted 6 lines · Pablo Neira Ayuso
Drivers do not register to netdev events to set up indirect blocks anymore. Remove
__flow_indr_block_cb_register()and__flow_indr_block_cb_unregister().The frontends set up the callbacks through
flow_indr_dev_setup_block()
Quoted 9 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Pablo Neira Ayuso <[email protected]>This commit failed to remove the prototypes from the header file:
include/net/flow_offload.hThanks for reporting.
I'm attaching a sketch, I will submit this formally later.
The sketch looks good to me.
Regards,
Jake
Quoted 15 lines · Pablo Neira Ayuso
Hi,
This patchset fixes the indirect
flow_blocksupport for the tc CT action offload. Please, note that this batch is probably slightly large for the net tree, however, I could not find a simple incremental fix.= The problem
The
nf_flow_table_indr_block_cb()function provides the tunnel netdevice and the indirectflow_blockdriver callback. From this tunnel netdevice, it is not possible to obtain the tc CTflow_block. Note that tc qdisc and netfilter backtrack from the tunnel netdevice to the tc block / netfilter chain to reach theflow_blockobject. This allows them to clean up the hardware offload rules if the tunnel device is removed.
Quoted 7 lines
<snip>
= About this patchset
This patchset aims to address the existing TC CT problem while simplifying the indirect
flow_blockinfrastructure. Saving 300 LoC in theflow_offloadcore and the drivers.
This might be a dumb question, but: what is the actual bug being fixed, that makes this patch series needed on net rather than net-next? From your description it sounds like a good and useful cleanup/refactor, but nonetheless still a clean-up, appropriate for net-next.
-ed
Pablo Neira Ayuso
maintainerQuoted 18 lines · Edward Cree
Quoted 17 lines · Pablo Neira Ayuso
Hi,
This patchset fixes the indirect
flow_blocksupport for the tc CT action offload. Please, note that this batch is probably slightly large for the net tree, however, I could not find a simple incremental fix.= The problem
The
nf_flow_table_indr_block_cb()function provides the tunnel netdevice and the indirectflow_blockdriver callback. From this tunnel netdevice, it is not possible to obtain the tc CTflow_block. Note that tc qdisc and netfilter backtrack from the tunnel netdevice to the tc block / netfilter chain to reach theflow_blockobject. This allows them to clean up the hardware offload rules if the tunnel device is removed.<snip>
Quoted 8 lines
Quoted 5 lines
= About this patchset
This patchset aims to address the existing TC CT problem while simplifying the indirect
flow_blockinfrastructure. Saving 300 LoC in theflow_offloadcore and the drivers.This might be a dumb question, but: what is the actual bug being fixed, that makes this patch series needed on net rather than net-next?
The TC CT action crashes the kernel with an indirect flow_block in place:
David Miller
maintainerFrom: Pablo Neira Ayuso <[email protected]>
Date: Fri, 15 May 2020 00:36:27 +0200
Quoted 3 lines
The TC CT action crashes the kernel with an indirect
flow_blockin place:
I've read over this patch set at least three times, and reread the header posting, and there is no clear indication that this patch series fixes a crash at all.
You need to be explicit about what bug this is fixing, in the commit messages and introduction posting, with quoted crash logs etc.