flow_offload hardware priority fixes
Subsystems (11)
Pablo Neira Ayuso
maintainercover flow_offload hardware priority fixes
Hi,
This patchset contains two updates for the flow_offload users:
1) Pass the major tc priority to drivers so they do not have to lshift it. This is a preparation patch for the fix coming in patch #2.
2) Set the hardware priority from the netfilter basechain priority, some drivers break when using the existing hardware priority number that is set to zero.
Please, apply, thank you.
Pablo Neira Ayuso (2):
net: sched: use major priority number as hardware priority
netfilter: nf_tables: map basechain priority to hardware priority
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c | 2 +- drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c | 2 +- drivers/net/ethernet/mscc/ocelot_flower.c | 12 +++--------- drivers/net/ethernet/netronome/nfp/flower/qos_conf.c | 2 +- drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c | 2 +- include/net/netfilter/nf_tables_offload.h | 2 ++ include/net/pkt_cls.h | 2 +- net/netfilter/nf_tables_api.c | 4 ++++ net/netfilter/nf_tables_offload.c | 18 +++++++++++++++--- 9 files changed, 29 insertions(+), 17 deletions(-)
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH 1/2 net: sched: use major priority number as hardware priority
tc transparently maps the software priority number to hardware. Update it to pass the major priority which is what most drivers expect. Update drivers too so they do not need to lshift the priority field of the flow_cls_common_offload object. The stmmac driver is an exception, since this code assumes the tc software priority is fine, therefore, lshift it just to be conservative.
---
v1: initial version.
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c | 2 +- drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c | 2 +- drivers/net/ethernet/mscc/ocelot_flower.c | 12 +++--------- drivers/net/ethernet/netronome/nfp/flower/qos_conf.c | 2 +- drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c | 2 +- include/net/pkt_cls.h | 2 +- 6 files changed, 8 insertions(+), 14 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +1 -1
@@ -3158,7 +3158,7 @@ mlx5e_flow_esw_attr_init(struct mlx5_esw_flow_attr *esw_attr, esw_attr->parse_attr = parse_attr; esw_attr->chain = f->common.chain_index; - esw_attr->prio =TC_H_MAJ (f->common.prio) >>16 ; + esw_attr->prio = f->common.prio; esw_attr->in_rep = in_rep; esw_attr->in_mdev = in_mdev;
drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c +1 -1
@@ -471,7 +471,7 @@ int mlxsw_sp_acl_rulei_commit(struct mlxsw_sp_acl_rule_info *rulei)void mlxsw_sp_acl_rulei_priority (struct mlxsw_sp_acl_rule_info *rulei,unsigned int priority) { - rulei->priority = priority >>16 ; + rulei->priority = priority; } void mlxsw_sp_acl_rulei_keymask_u32(struct mlxsw_sp_acl_rule_info *rulei,
drivers/net/ethernet/mscc/ocelot_flower.c +3 -9
@@ -13,12 +13,6 @@ struct ocelot_port_block {struct ocelot_port *port; }; -static u16 get_prio (u32 prio) -{ -/* prio starts from 0x1000 while the ids starts from 0 */ -return prio >>16 ; -} -static int ocelot_flower_parse_action (struct flow_cls_offload *f,struct ocelot_ace_rule *rule) {
@@ -168,7 +162,7 @@ static int ocelot_flower_parse(struct flow_cls_offload *f, } finished_key_parsing: - ocelot_rule->prio =get_prio (f->common.prio); + ocelot_rule->prio = f->common.prio; ocelot_rule->id = f->cookie;return ocelot_flower_parse_action (f, ocelot_rule); }
@@ -218,7 +212,7 @@ static int ocelot_flower_destroy(struct flow_cls_offload *f,struct ocelot_ace_rule rule;int ret; - rule.prio =get_prio (f->common.prio); + rule.prio = f->common.prio; rule.port = port_block->port; rule.id = f->cookie;
@@ -236,7 +230,7 @@ static int ocelot_flower_stats_update(struct flow_cls_offload *f,struct ocelot_ace_rule rule;int ret; - rule.prio =get_prio (f->common.prio); + rule.prio = f->common.prio; rule.port = port_block->port; rule.id = f->cookie; ret =ocelot_ace_rule_stats_update (&rule);
drivers/net/ethernet/netronome/nfp/flower/qos_conf.c +1 -1
@@ -93,7 +93,7 @@ nfp_flower_install_rate_limiter(struct nfp_app *app, struct net_device *netdev,return -EOPNOTSUPP ; } -if (flow->common.prio != (1 <<16 )) { +if (flow->common.prio !=1 ) {NL_SET_ERR_MSG_MOD (extack,"unsupported offload: qos rate limit offload requires highest priority" );return -EOPNOTSUPP ; }
drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c +1 -1
@@ -94,7 +94,7 @@ static int tc_fill_entry(struct stmmac_priv *priv,struct stmmac_tc_entry *entry, *frag =NULL ;struct tc_u32_sel *sel = cls->knode.sel;u32 off, data, mask, real_off, rem; -u32 prio = cls->common.prio; +u32 prio = cls->common.prio <<16 ;int ret;/* Only 1 match per entry */
include/net/pkt_cls.h +1 -1
@@ -646,7 +646,7 @@ tc_cls_common_offload_init(struct flow_cls_common_offload *cls_common, { cls_common->chain_index = tp->chain->index; cls_common->protocol = tp->protocol; - cls_common->prio = tp->prio; + cls_common->prio = tp->prio >>16 ;if (tc_skip_sw (flags) || flags &TCA_CLS_FLAGS_VERBOSE ) cls_common->extack = extack; }
Signature · 1 line
2.11.0
Pablo Neira Ayuso
maintainerPATCH v3 2/2 netfilter: nf_tables: map basechain priority to hardware priority
In reply to Pablo Neira AyusoThis patch adds initial support for offloading basechains using the priority range from -32767 to 32767. This is restricting the netfilter priority range to 16-bit integer since this is what most drivers assume so far from tc.
The software to hardware priority mapping is not exposed to userspace. Hence, it should be possible to extend this range of supported priorities later on once drivers are updated to support for 32-bit integer priorities.
---
v3: update mapping not to skip tc priority space, each subsystem has
its own priority space. - Jakub Kicinski.
include/net/netfilter/nf_tables_offload.h | 2 ++ net/netfilter/nf_tables_api.c | 4 ++++ net/netfilter/nf_tables_offload.c | 18 +++++++++++++++--- 3 files changed, 21 insertions(+), 3 deletions(-)
include/net/netfilter/nf_tables_offload.h +2 -0
@@ -73,4 +73,6 @@ int nft_flow_rule_offload_commit(struct net *net); (__reg)->key = __key; \memset (&(__reg)->mask,0xff , (__reg)->len); +u32 nft_chain_offload_priority (struct nft_base_chain *basechain); +#endif
net/netfilter/nf_tables_api.c +4 -0
@@ -1662,6 +1662,10 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask, chain->flags |=NFT_BASE_CHAIN | flags; basechain->policy =NF_ACCEPT ; +if (chain->flags &NFT_CHAIN_HW_OFFLOAD && + !nft_chain_offload_priority (basechain)) +return -EOPNOTSUPP ; +flow_block_init (&basechain->flow_block); }else { chain =kzalloc (sizeof (*chain),GFP_KERNEL );
net/netfilter/nf_tables_offload.c +15 -3
@@ -103,10 +103,11 @@ void nft_offload_update_dependency(struct nft_offload_ctx *ctx, }static void nft_flow_offload_common_init (struct flow_cls_common_offload *common, -__be16 proto, -struct netlink_ext_ack *extack) +__be16 proto,int priority, +struct netlink_ext_ack *extack) { common->protocol = proto; + common->prio = priority; common->extack = extack; }
@@ -124,6 +125,15 @@ static int nft_setup_cb_call(struct nft_base_chain *basechain,return 0 ; } +u32 nft_chain_offload_priority (struct nft_base_chain *basechain) +{ +if (basechain->ops.priority <SHRT_MIN || + basechain->ops.priority >SHRT_MAX ) +return 0 ; + +return basechain->ops.priority +abs (SHRT_MIN ); +} +static int nft_flow_offload_rule (struct nft_trans *trans,enum flow_cls_command command) {
@@ -142,7 +152,9 @@ static int nft_flow_offload_rule(struct nft_trans *trans,if (flow) proto = flow->proto; -nft_flow_offload_common_init (&cls_flow.common, proto, &extack); +nft_flow_offload_common_init (&cls_flow.common, proto, +nft_chain_offload_priority (basechain), + &extack); cls_flow.command = command; cls_flow.cookie = (unsigned long ) rule;if (flow)
Signature · 1 line
2.11.0
Quoted 1 line · Pablo Neira Ayuso
Please, apply, thank you.
I'm still waiting for a reply.
Perhaps since Pablo doesn't want to talk to me someone else can explain to me why we want to seemingly diverge from the software model?
Pablo Neira Ayuso
maintainerHi Jakub,
If the user specifies 'pref' in the new rule, then tc checks if there is a tcf_proto object that matches this priority. If the tcf_proto object does not exist, tc creates a tcf_proto object and it adds the new rule to this tcf_proto.
In cls_flower, each tcf_proto only stores one single rule, so if the user tries to add another rule with the same 'pref', cls_flower returns EEXIST.
I'll prepare a new patchset not to map the priority to the netfilter basechain priority, instead the rule priority will be internally allocated for each new rule.
Thanks for your feedback.
Quoted 10 lines · Pablo Neira Ayuso
Hi Jakub,
If the user specifies 'pref' in the new rule, then tc checks if there is a
tcf_protoobject that matches this priority. If thetcf_protoobject does not exist, tc creates atcf_protoobject and it adds the new rule to thistcf_proto.In
cls_flower, eachtcf_protoonly stores one single rule, so if the user tries to add another rule with the same 'pref',cls_flowerreturns EEXIST.
😳
So you're saying this doesn't work?
ip link add type dummy
tc qdisc add dev dummy0 clsact
tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::1 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::2 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::3 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::4 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::5 action drop
tc filter show dev dummy0 ingress
filter protocol ipv6 pref 123 flower chain 0 filter protocol ipv6 pref 123 flower chain 0 handle 0x1
eth_type ipv6
src_ip 1111::1
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 1 ref 1 bind 1
filter protocol ipv6 pref 123 flower chain 0 handle 0x2
eth_type ipv6
src_ip 1111::2
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 2 ref 1 bind 1
filter protocol ipv6 pref 123 flower chain 0 handle 0x3
eth_type ipv6
src_ip 1111::3
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 3 ref 1 bind 1
filter protocol ipv6 pref 123 flower chain 0 handle 0x4
eth_type ipv6
src_ip 1111::4
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 4 ref 1 bind 1
filter protocol ipv6 pref 123 flower chain 0 handle 0x5
eth_type ipv6
src_ip 1111::5
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 5 ref 1 bind 1
Quoted 3 lines
I'll prepare a new patchset not to map the priority to the netfilter basechain priority, instead the rule priority will be internally allocated for each new rule.
In which you're adding fake priorities to rules, AFAICT, and continue to baffle me.
Pablo Neira Ayuso
maintainerHi Jakub,
Quoted 12 lines · Jakub Kicinski
Quoted 10 lines · Pablo Neira Ayuso
Hi Jakub,
If the user specifies 'pref' in the new rule, then tc checks if there is a
tcf_protoobject that matches this priority. If thetcf_protoobject does not exist, tc creates atcf_protoobject and it adds the new rule to thistcf_proto.In
cls_flower, eachtcf_protoonly stores one single rule, so if the user tries to add another rule with the same 'pref',cls_flowerreturns EEXIST.
Quoted 6 lines
So you're saying this doesn't work?
ip link add type dummy
tc qdisc add dev dummy0 clsact
tc filter add dev dummy0 ingress protocol ipv6 prio 123 flowersrc_ip1111::1 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flowersrc_ip1111::2 action drop
This works indeed as you describe.
I'll go back to the original netfilter basechain priority patch then:
https://patchwork.ozlabs.org/patch/1140412/
That patch removed the reference to tcf_auto_prio() already, please let me know if you have any more specific update you would like to see on that patch.
Thanks.
Quoted 3 lines · Pablo Neira Ayuso
That patch removed the reference to
tcf_auto_prio()already, please let me know if you have any more specific update you would like to see on that patch.
Please explain why the artificial priorities are needed at all. Hardware should order tables based on table type - ethtool, TC, nft. As I mentioned in the first email, and unless you can make a strong case against that.
Within those tables we should follow the same ordering rules as we do in software (modulo ethtool but ordering is pretty clear there).
Pablo Neira Ayuso
maintainerHi Jakub,
Quoted 11 lines · Jakub Kicinski
Quoted 3 lines · Pablo Neira Ayuso
That patch removed the reference to
tcf_auto_prio()already, please let me know if you have any more specific update you would like to see on that patch.Please explain why the artificial priorities are needed at all. Hardware should order tables based on table type - ethtool, TC, nft. As I mentioned in the first email, and unless you can make a strong case against that.
Within those tables we should follow the same ordering rules as we do in software (modulo ethtool but ordering is pretty clear there).
The idea is that every subsystem (ethtool, tc, nf) sets up/binds its own flow_block object. And each flow_block object has its own priority range space. So whatever priority the user specifies only applies to the specific subsystem. Drivers still need to be updated to support for more than one flow_block/subsystem binding at this stage though.
Quoted 4 lines · Pablo Neira Ayuso
The idea is that every subsystem (ethtool, tc, nf) sets up/binds its own
flow_blockobject. And eachflow_blockobject has its own priority range space. So whatever priority the user specifies only applies to the specific subsystem.
Right, okay so that part is pretty obvious but thanks for spelling it out.
Are you also agreeing that priorities of blocks, not rules within a block are dictated by the order of processing within the kernel? IOW TC blocks are _always_ before nft blocks?
Quoted 2 lines
Drivers still need to be updated to support for more than one flow_block/subsystem binding at this stage though.
Pablo Neira Ayuso
maintainerQuoted 6 lines · Jakub Kicinski
Quoted 4 lines · Pablo Neira Ayuso
The idea is that every subsystem (ethtool, tc, nf) sets up/binds its own
flow_blockobject. And eachflow_blockobject has its own priority range space. So whatever priority the user specifies only applies to the specific subsystem.
Quoted 6 lines
Right, okay so that part is pretty obvious but thanks for spelling it out.
Are you also agreeing that priorities of blocks, not rules within a block are dictated by the order of processing within the kernel? IOW TC blocks are _always_ before nft blocks?
yes.