Skip to contentSkip to conversation

flow_offload hardware priority fixes

PATCH Superseded Pablo Neira Ayuso 2 participants 11 messages Aug 1, 2019 – Aug 6, 2019

Subsystems (11)

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Pablo Neira Ayuso

author maintainer
#0 lore raw

cover flow_offload hardware priority fixes

Hi,

This patchset contains two updates for the flow_offload users:

1) Pass the major tc priority to drivers so they do not have to lshift it. This is a preparation patch for the fix coming in patch #2.

2) Set the hardware priority from the netfilter basechain priority, some drivers break when using the existing hardware priority number that is set to zero.

Please, apply, thank you.

Pablo Neira Ayuso (2):
net: sched: use major priority number as hardware priority
netfilter: nf_tables: map basechain priority to hardware priority

 drivers/net/ethernet/mellanox/mlx5/core/en_tc.c      |  2 +-
 drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c   |  2 +-
 drivers/net/ethernet/mscc/ocelot_flower.c            | 12 +++---------
 drivers/net/ethernet/netronome/nfp/flower/qos_conf.c |  2 +-
 drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c      |  2 +-
 include/net/netfilter/nf_tables_offload.h            |  2 ++
 include/net/pkt_cls.h                                |  2 +-
 net/netfilter/nf_tables_api.c                        |  4 ++++
 net/netfilter/nf_tables_offload.c                    | 18 +++++++++++++++---
 9 files changed, 29 insertions(+), 17 deletions(-)
Signature · 1 line
2.11.0
author maintainer
#1 lore raw

PATCH 1/2 net: sched: use major priority number as hardware priority

tc transparently maps the software priority number to hardware. Update it to pass the major priority which is what most drivers expect. Update drivers too so they do not need to lshift the priority field of the flow_cls_common_offload object. The stmmac driver is an exception, since this code assumes the tc software priority is fine, therefore, lshift it just to be conservative.

---
v1: initial version.

 drivers/net/ethernet/mellanox/mlx5/core/en_tc.c      |  2 +-
 drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c   |  2 +-
 drivers/net/ethernet/mscc/ocelot_flower.c            | 12 +++---------
 drivers/net/ethernet/netronome/nfp/flower/qos_conf.c |  2 +-
 drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c      |  2 +-
 include/net/pkt_cls.h                                |  2 +-
 6 files changed, 8 insertions(+), 14 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +1 -1
index cc096f6011d9..744c0c640c10 100644
@@ -3158,7 +3158,7 @@ mlx5e_flow_esw_attr_init(struct mlx5_esw_flow_attr *esw_attr,
 
 	esw_attr->parse_attr = parse_attr;
 	esw_attr->chain = f->common.chain_index;
-	esw_attr->prio = TC_H_MAJ(f->common.prio) >> 16;
+	esw_attr->prio = f->common.prio;
 
 	esw_attr->in_rep = in_rep;
 	esw_attr->in_mdev = in_mdev;
drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c +1 -1
index e8ac90564dbe..84a87d059333 100644
@@ -471,7 +471,7 @@ int mlxsw_sp_acl_rulei_commit(struct mlxsw_sp_acl_rule_info *rulei)
 void mlxsw_sp_acl_rulei_priority(struct mlxsw_sp_acl_rule_info *rulei,
 				 unsigned int priority)
 {
-	rulei->priority = priority >> 16;
+	rulei->priority = priority;
 }
 
 void mlxsw_sp_acl_rulei_keymask_u32(struct mlxsw_sp_acl_rule_info *rulei,
drivers/net/ethernet/mscc/ocelot_flower.c +3 -9
index 59487d446a09..b894bc0c9c16 100644
@@ -13,12 +13,6 @@ struct ocelot_port_block {
 	struct ocelot_port *port;
 };
 
-static u16 get_prio(u32 prio)
-{
-	/* prio starts from 0x1000 while the ids starts from 0 */
-	return prio >> 16;
-}
-
 static int ocelot_flower_parse_action(struct flow_cls_offload *f,
 				      struct ocelot_ace_rule *rule)
 {
@@ -168,7 +162,7 @@ static int ocelot_flower_parse(struct flow_cls_offload *f,
 	}
 
 finished_key_parsing:
-	ocelot_rule->prio = get_prio(f->common.prio);
+	ocelot_rule->prio = f->common.prio;
 	ocelot_rule->id = f->cookie;
 	return ocelot_flower_parse_action(f, ocelot_rule);
 }
@@ -218,7 +212,7 @@ static int ocelot_flower_destroy(struct flow_cls_offload *f,
 	struct ocelot_ace_rule rule;
 	int ret;
 
-	rule.prio = get_prio(f->common.prio);
+	rule.prio = f->common.prio;
 	rule.port = port_block->port;
 	rule.id = f->cookie;
 
@@ -236,7 +230,7 @@ static int ocelot_flower_stats_update(struct flow_cls_offload *f,
 	struct ocelot_ace_rule rule;
 	int ret;
 
-	rule.prio = get_prio(f->common.prio);
+	rule.prio = f->common.prio;
 	rule.port = port_block->port;
 	rule.id = f->cookie;
 	ret = ocelot_ace_rule_stats_update(&rule);
drivers/net/ethernet/netronome/nfp/flower/qos_conf.c +1 -1
index 86e968cd5ffd..124a43dc136a 100644
@@ -93,7 +93,7 @@ nfp_flower_install_rate_limiter(struct nfp_app *app, struct net_device *netdev,
 		return -EOPNOTSUPP;
 	}
 
-	if (flow->common.prio != (1 << 16)) {
+	if (flow->common.prio != 1) {
 		NL_SET_ERR_MSG_MOD(extack, "unsupported offload: qos rate limit offload requires highest priority");
 		return -EOPNOTSUPP;
 	}
drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c +1 -1
index 58ea18af9813..5cd040215469 100644
@@ -94,7 +94,7 @@ static int tc_fill_entry(struct stmmac_priv *priv,
 	struct stmmac_tc_entry *entry, *frag = NULL;
 	struct tc_u32_sel *sel = cls->knode.sel;
 	u32 off, data, mask, real_off, rem;
-	u32 prio = cls->common.prio;
+	u32 prio = cls->common.prio << 16;
 	int ret;
 
 	/* Only 1 match per entry */
include/net/pkt_cls.h +1 -1
index e429809ca90d..98be18ef1ed3 100644
@@ -646,7 +646,7 @@ tc_cls_common_offload_init(struct flow_cls_common_offload *cls_common,
 {
 	cls_common->chain_index = tp->chain->index;
 	cls_common->protocol = tp->protocol;
-	cls_common->prio = tp->prio;
+	cls_common->prio = tp->prio >> 16;
 	if (tc_skip_sw(flags) || flags & TCA_CLS_FLAGS_VERBOSE)
 		cls_common->extack = extack;
 }
Signature · 1 line
2.11.0
author maintainer
#2 lore raw

PATCH v3 2/2 netfilter: nf_tables: map basechain priority to hardware priority

In reply to Pablo Neira Ayuso

This patch adds initial support for offloading basechains using the priority range from -32767 to 32767. This is restricting the netfilter priority range to 16-bit integer since this is what most drivers assume so far from tc.

The software to hardware priority mapping is not exposed to userspace. Hence, it should be possible to extend this range of supported priorities later on once drivers are updated to support for 32-bit integer priorities.

---
v3: update mapping not to skip tc priority space, each subsystem has
its own priority space. - Jakub Kicinski.

 include/net/netfilter/nf_tables_offload.h |  2 ++
 net/netfilter/nf_tables_api.c             |  4 ++++
 net/netfilter/nf_tables_offload.c         | 18 +++++++++++++++---
 3 files changed, 21 insertions(+), 3 deletions(-)
include/net/netfilter/nf_tables_offload.h +2 -0
index 3196663a10e3..3c31e9d55028 100644
@@ -73,4 +73,6 @@ int nft_flow_rule_offload_commit(struct net *net);
 	(__reg)->key		= __key;				\
 	memset(&(__reg)->mask, 0xff, (__reg)->len);
 
+u32 nft_chain_offload_priority(struct nft_base_chain *basechain);
+
 #endif
net/netfilter/nf_tables_api.c +4 -0
index 605a7cfe7ca7..9cf0fecf5cb9 100644
@@ -1662,6 +1662,10 @@ static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
 
 		chain->flags |= NFT_BASE_CHAIN | flags;
 		basechain->policy = NF_ACCEPT;
+		if (chain->flags & NFT_CHAIN_HW_OFFLOAD &&
+		    !nft_chain_offload_priority(basechain))
+			return -EOPNOTSUPP;
+
 		flow_block_init(&basechain->flow_block);
 	} else {
 		chain = kzalloc(sizeof(*chain), GFP_KERNEL);
net/netfilter/nf_tables_offload.c +15 -3
index 64f5fd5f240e..81d636fac571 100644
@@ -103,10 +103,11 @@ void nft_offload_update_dependency(struct nft_offload_ctx *ctx,
 }
 
 static void nft_flow_offload_common_init(struct flow_cls_common_offload *common,
-					 __be16 proto,
-					struct netlink_ext_ack *extack)
+					 __be16 proto, int priority,
+					 struct netlink_ext_ack *extack)
 {
 	common->protocol = proto;
+	common->prio = priority;
 	common->extack = extack;
 }
 
@@ -124,6 +125,15 @@ static int nft_setup_cb_call(struct nft_base_chain *basechain,
 	return 0;
 }
 
+u32 nft_chain_offload_priority(struct nft_base_chain *basechain)
+{
+	if (basechain->ops.priority < SHRT_MIN ||
+	    basechain->ops.priority > SHRT_MAX)
+		return 0;
+
+	return basechain->ops.priority + abs(SHRT_MIN);
+}
+
 static int nft_flow_offload_rule(struct nft_trans *trans,
 				 enum flow_cls_command command)
 {
@@ -142,7 +152,9 @@ static int nft_flow_offload_rule(struct nft_trans *trans,
 	if (flow)
 		proto = flow->proto;
 
-	nft_flow_offload_common_init(&cls_flow.common, proto, &extack);
+	nft_flow_offload_common_init(&cls_flow.common, proto,
+				     nft_chain_offload_priority(basechain),
+				     &extack);
 	cls_flow.command = command;
 	cls_flow.cookie = (unsigned long) rule;
 	if (flow)
Signature · 1 line
2.11.0
#3 lore raw
In reply to Pablo Neira Ayuso
Quoted 1 line · Pablo Neira Ayuso

Please, apply, thank you.

I'm still waiting for a reply.

Perhaps since Pablo doesn't want to talk to me someone else can explain to me why we want to seemingly diverge from the software model?

author maintainer
#4 lore raw

Hi Jakub,

If the user specifies 'pref' in the new rule, then tc checks if there is a tcf_proto object that matches this priority. If the tcf_proto object does not exist, tc creates a tcf_proto object and it adds the new rule to this tcf_proto.

In cls_flower, each tcf_proto only stores one single rule, so if the user tries to add another rule with the same 'pref', cls_flower returns EEXIST.

I'll prepare a new patchset not to map the priority to the netfilter basechain priority, instead the rule priority will be internally allocated for each new rule.

Thanks for your feedback.

#5 lore raw
Quoted 10 lines · Pablo Neira Ayuso

Hi Jakub,

If the user specifies 'pref' in the new rule, then tc checks if there is a tcf_proto object that matches this priority. If the tcf_proto object does not exist, tc creates a tcf_proto object and it adds the new rule to this tcf_proto.

In cls_flower, each tcf_proto only stores one single rule, so if the user tries to add another rule with the same 'pref', cls_flower returns EEXIST.

😳

So you're saying this doesn't work?

ip link add type dummy
tc qdisc add dev dummy0 clsact
tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::1 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::2 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::3 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::4 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::5 action drop

tc filter show dev dummy0 ingress

filter protocol ipv6 pref 123 flower chain 0 filter protocol ipv6 pref 123 flower chain 0 handle 0x1
eth_type ipv6
src_ip 1111::1
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 1 ref 1 bind 1

filter protocol ipv6 pref 123 flower chain 0 handle 0x2
eth_type ipv6
src_ip 1111::2
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 2 ref 1 bind 1

filter protocol ipv6 pref 123 flower chain 0 handle 0x3
eth_type ipv6
src_ip 1111::3
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 3 ref 1 bind 1

filter protocol ipv6 pref 123 flower chain 0 handle 0x4
eth_type ipv6
src_ip 1111::4
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 4 ref 1 bind 1

filter protocol ipv6 pref 123 flower chain 0 handle 0x5
eth_type ipv6
src_ip 1111::5
not_in_hw
action order 1: gact action drop
random type none pass val 0
index 5 ref 1 bind 1

Quoted 3 lines

I'll prepare a new patchset not to map the priority to the netfilter basechain priority, instead the rule priority will be internally allocated for each new rule.

In which you're adding fake priorities to rules, AFAICT, and continue to baffle me.

author maintainer
#6 lore raw

Hi Jakub,

Quoted 12 lines · Jakub Kicinski
Quoted 10 lines · Pablo Neira Ayuso

Hi Jakub,

If the user specifies 'pref' in the new rule, then tc checks if there is a tcf_proto object that matches this priority. If the tcf_proto object does not exist, tc creates a tcf_proto object and it adds the new rule to this tcf_proto.

In cls_flower, each tcf_proto only stores one single rule, so if the user tries to add another rule with the same 'pref', cls_flower returns EEXIST.

Quoted 6 lines

So you're saying this doesn't work?

ip link add type dummy
tc qdisc add dev dummy0 clsact
tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::1 action drop tc filter add dev dummy0 ingress protocol ipv6 prio 123 flower src_ip 1111::2 action drop

This works indeed as you describe.

I'll go back to the original netfilter basechain priority patch then:

https://patchwork.ozlabs.org/patch/1140412/

That patch removed the reference to tcf_auto_prio() already, please let me know if you have any more specific update you would like to see on that patch.

Thanks.

#7 lore raw
Quoted 3 lines · Pablo Neira Ayuso

That patch removed the reference to tcf_auto_prio() already, please let me know if you have any more specific update you would like to see on that patch.

Please explain why the artificial priorities are needed at all. Hardware should order tables based on table type - ethtool, TC, nft. As I mentioned in the first email, and unless you can make a strong case against that.
Within those tables we should follow the same ordering rules as we do in software (modulo ethtool but ordering is pretty clear there).

author maintainer
#8 lore raw

Hi Jakub,

Quoted 11 lines · Jakub Kicinski
Quoted 3 lines · Pablo Neira Ayuso

That patch removed the reference to tcf_auto_prio() already, please let me know if you have any more specific update you would like to see on that patch.

Please explain why the artificial priorities are needed at all. Hardware should order tables based on table type - ethtool, TC, nft. As I mentioned in the first email, and unless you can make a strong case against that.
Within those tables we should follow the same ordering rules as we do in software (modulo ethtool but ordering is pretty clear there).

The idea is that every subsystem (ethtool, tc, nf) sets up/binds its own flow_block object. And each flow_block object has its own priority range space. So whatever priority the user specifies only applies to the specific subsystem. Drivers still need to be updated to support for more than one flow_block/subsystem binding at this stage though.

#9 lore raw
Quoted 4 lines · Pablo Neira Ayuso

The idea is that every subsystem (ethtool, tc, nf) sets up/binds its own flow_block object. And each flow_block object has its own priority range space. So whatever priority the user specifies only applies to the specific subsystem.

Right, okay so that part is pretty obvious but thanks for spelling it out.

Are you also agreeing that priorities of blocks, not rules within a block are dictated by the order of processing within the kernel? IOW TC blocks are _always_ before nft blocks?

Quoted 2 lines

Drivers still need to be updated to support for more than one flow_block/subsystem binding at this stage though.

author maintainer
#10 lore raw
Quoted 6 lines · Jakub Kicinski
Quoted 4 lines · Pablo Neira Ayuso

The idea is that every subsystem (ethtool, tc, nf) sets up/binds its own flow_block object. And each flow_block object has its own priority range space. So whatever priority the user specifies only applies to the specific subsystem.

Quoted 6 lines

Right, okay so that part is pretty obvious but thanks for spelling it out.

Are you also agreeing that priorities of blocks, not rules within a block are dictated by the order of processing within the kernel? IOW TC blocks are _always_ before nft blocks?

yes.