Port-based priority on DSA switches using tc-matchall
Subsystems (4)
Vladimir Oltean
maintainercover Port-based priority on DSA switches using tc-matchall
From: Vladimir Oltean <[email protected]>
This is a proposal for configuring the port-based default priority on switch ports using tc-matchall and skbedit priority. Comments welcome.
Vladimir Oltean (2):
net: dsa: allow setting port-based QoS priority using tc matchall
skbedit
net: dsa: felix: offload port priority
drivers/net/dsa/ocelot/felix.c | 15 +++++++ include/net/dsa.h | 8 ++++ net/dsa/slave.c | 72 ++++++++++++++++++++++++++++++++++ 3 files changed, 95 insertions(+)
Signature · 1 line
2.25.1
Vladimir Oltean
maintainerRFC PATCH 1/2 net: dsa: allow setting port-based QoS priority using tc matchall skbedit
From: Vladimir Oltean <[email protected]>
In Time Sensitive Networking it is a common and simple use case to configure switches to give all traffic from an attached station the same priority, without requiring those stations to use VLAN PCP or IP DSCP to signal the priority that they want. Many pieces of hardware support this feature via a port-based default priority. We can model this in Linux through a matchall action on the ingress qdisc of the port, plus a skbedit priority action with the desired priority.
--- include/net/dsa.h | 8 ++++++ net/dsa/slave.c | 72 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+)
include/net/dsa.h +8 -0
@@ -155,6 +155,7 @@ struct dsa_switch_tree {enum dsa_port_mall_action_type {DSA_PORT_MALL_MIRROR ,DSA_PORT_MALL_POLICER , +DSA_PORT_MALL_SKBEDIT , };/* TC mirroring entry */
@@ -169,6 +170,10 @@ struct dsa_mall_policer_tc_entry {u64 rate_bytes_per_sec; }; +struct dsa_mall_skbedit_tc_entry { +int priority; +}; +/* TC matchall entry */ struct dsa_mall_tc_entry {struct list_head list;
@@ -177,6 +182,7 @@ struct dsa_mall_tc_entry {union {struct dsa_mall_mirror_tc_entry mirror;struct dsa_mall_policer_tc_entry policer; +struct dsa_mall_skbedit_tc_entry skbedit; }; };
@@ -612,6 +618,8 @@ struct dsa_switch_ops {int (*port_policer_add)(struct dsa_switch *ds,int port,struct dsa_mall_policer_tc_entry *policer);void (*port_policer_del)(struct dsa_switch *ds,int port); +int (*port_priority_set)(struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit);int (*port_setup_tc)(struct dsa_switch *ds,int port,enum tc_setup_type type,void *type_data);
net/dsa/slave.c +72 -0
90 lines not on this page · raw message
Signature · 1 line
2.25.1
Andrew Lunn
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Quoted 9 lines · Vladimir Oltean
From: Vladimir Oltean <[email protected]>
In Time Sensitive Networking it is a common and simple use case to configure switches to give all traffic from an attached station the same priority, without requiring those stations to use VLAN PCP or IP DSCP to signal the priority that they want. Many pieces of hardware support this feature via a port-based default priority. We can model this in Linux through a matchall action on the ingress qdisc of the port, plus a skbedit priority action with the desired priority.
The mv88e6xxx has something similar. There is a bit to enable this feature, as well as the priority the feature should have. I think that then takes a value in the range of 0 to 4, but i could be remembering it wrongly.
include/net/dsa.h struct dsa_switch_ops { · line 622
Quoted 2 lines
include/net/dsa.h +2
-0+int (*port_priority_set)(struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit);
The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Andrew
Vladimir Oltean
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Quoted 7 lines · Andrew Lunn
include/net/dsa.h struct dsa_switch_ops { · line 622
Quoted 2 lines · Vladimir Oltean
include/net/dsa.h +2
-0+int (*port_priority_set)(struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
Andrew Lunn
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Quoted 9 lines · Vladimir Oltean
Quoted 7 lines · Andrew Lunn
include/net/dsa.h struct dsa_switch_ops { · line 622
Quoted 2 lines · Vladimir Oltean
include/net/dsa.h +2
-0+int (*port_priority_set)(struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Quoted 6 lines
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
That does not work for mv88e6xxx. Its default setup, if i remember correctly, is it looks at the TOS bits to determine priority classes. So in its default state, it is using all the available traffic classes. It can also be configured to look at the VLAN priority, or the TCAM can set the priority class, or there is a per port default priority, which is what you are describing here. There are bits to select which of these happen on ingress, on a per port basis.
So setting the port priority to 0 means setting the priority of zero. It does not mean go back to the default prioritisation scheme.
I guess any switch which has a range of options for prioritisation selection will have a similar problem. It defaults to something, probably something a bit smarter than everything goes to traffic class 0.
Andrew
Vladimir Oltean
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Hi Andrew,
Quoted 29 lines · Andrew Lunn
Quoted 9 lines · Vladimir Oltean
Quoted 7 lines · Andrew Lunn
Quoted 2 lines · Vladimir Oltean
+ int (*port_priority_set)(struct
dsa_switch*ds, int port,
+ structdsa_mall_skbedit_tc_entry*skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Quoted 6 lines
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
That does not work for mv88e6xxx. Its default setup, if i remember correctly, is it looks at the TOS bits to determine priority classes. So in its default state, it is using all the available traffic classes. It can also be configured to look at the VLAN priority, or the TCAM can set the priority class, or there is a per port default priority, which is what you are describing here. There are bits to select which of these happen on ingress, on a per port basis.
So setting the port priority to 0 means setting the priority of zero. It does not mean go back to the default prioritisation scheme.
Quoted 6 lines
I guess any switch which has a range of options for prioritisation selection will have a similar problem. It defaults to something, probably something a bit smarter than everything goes to traffic class 0.
Andrew
I was going through my old patches, and re-reading this conversation, it appears one of us is misunderstanding something.
I looked at some Marvell datasheet and it has a similar QoS classification pipeline to Vitesse switches. There is a port-based default priority which can be overridden by IP DSCP, VLAN PCP, or advanced QoS classification (TCAM).
The proposal I had was to configure the default port priority using tc matchall skbedit priority. Advanced QoS classification would then be expressed as tc-flower filters with a higher precedence than the matchall (basically the "catchall"). PCP and DSCP, I don't know if that can be expressed cleanly using tc. I think there's something in the dcb ops, but I haven't studied that too deeply.
Anyway, I don't exactly understand your point, that an add/del is in any way better than a "set". Even for Marvell, what I'm proposing here would translate in a "set to 0" on "del" anyway. That's why this patch set is RFC. I don't know if there's a better way to express a port-based default priority than a matchall rule having the lowest precedence.
Re: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Quoted 47 lines · Vladimir Oltean
Hi Andrew,
Quoted 29 lines · Andrew Lunn
Quoted 9 lines · Vladimir Oltean
Quoted 7 lines · Andrew Lunn
Quoted 2 lines · Vladimir Oltean
+ int (*port_priority_set)(struct
dsa_switch*ds, int port,
+ structdsa_mall_skbedit_tc_entry*skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Quoted 6 lines
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
That does not work for mv88e6xxx. Its default setup, if i remember correctly, is it looks at the TOS bits to determine priority classes. So in its default state, it is using all the available traffic classes. It can also be configured to look at the VLAN priority, or the TCAM can set the priority class, or there is a per port default priority, which is what you are describing here. There are bits to select which of these happen on ingress, on a per port basis.
So setting the port priority to 0 means setting the priority of zero. It does not mean go back to the default prioritisation scheme.
Quoted 6 lines
I guess any switch which has a range of options for prioritisation selection will have a similar problem. It defaults to something, probably something a bit smarter than everything goes to traffic class 0.
Andrew
I was going through my old patches, and re-reading this conversation, it appears one of us is misunderstanding something.
I looked at some Marvell datasheet and it has a similar QoS classification pipeline to Vitesse switches. There is a port-based default priority which can be overridden by IP DSCP, VLAN PCP, or advanced QoS classification (TCAM).
Quoted 6 lines
The proposal I had was to configure the default port priority using tc matchall skbedit priority. Advanced QoS classification would then be expressed as tc-flower filters with a higher precedence than the matchall (basically the "catchall"). PCP and DSCP, I don't know if that can be expressed cleanly using tc. I think there's something in the dcb ops, but I haven't studied that too deeply.
In 802.1Q-2014, port-default priority is handled as APP entries matching on EtherType of 0. (See Table D-9.) Those are "default priority. For use when priority is not otherwise specified".
So DCB ops just handle these as APP entries. Dunno what DSA does. In mlxsw, we call dcb_ieee_getapp_default_prio_mask() when the DCP set_app hook fires to find the relevant entries and get the priority bitmask.
Now I don't understand DSA at all, but given a chip with fancy defaults, for the DCB interface in particular, it would make sense to me to have two ops. As long as there are default-prio entries, a "set default priority" op would get invoked with the highest configured default priority. When the last entry disappears, an "unset" op would be called.
Not sure what DSA does with ACLs, but it's not clear to me how TC-based prioritization rules coexist with full blown ACLs. I suppose the prio stuff could live on chain 0 and all actions would be skbedit prio pipe goto chain 1 or something. And goto chain 0 is forbidden, because chain 0 is special. Or maybe the prioritization stuff lives on a root qdisc (but no, we need it for ingress packets...) One way or another it looks hairy to dissect and offload accurately IMHO.
Vladimir Oltean
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Hi Petr,
Quoted 57 lines · Petr Machata
Quoted 47 lines · Vladimir Oltean
Hi Andrew,
Quoted 29 lines · Andrew Lunn
Quoted 9 lines · Vladimir Oltean
Quoted 7 lines · Andrew Lunn
Quoted 2 lines · Vladimir Oltean
+ int (*port_priority_set)(struct
dsa_switch*ds, int port,
+ structdsa_mall_skbedit_tc_entry*skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Quoted 6 lines
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
That does not work for mv88e6xxx. Its default setup, if i remember correctly, is it looks at the TOS bits to determine priority classes. So in its default state, it is using all the available traffic classes. It can also be configured to look at the VLAN priority, or the TCAM can set the priority class, or there is a per port default priority, which is what you are describing here. There are bits to select which of these happen on ingress, on a per port basis.
So setting the port priority to 0 means setting the priority of zero. It does not mean go back to the default prioritisation scheme.
Quoted 6 lines
I guess any switch which has a range of options for prioritisation selection will have a similar problem. It defaults to something, probably something a bit smarter than everything goes to traffic class 0.
Andrew
I was going through my old patches, and re-reading this conversation, it appears one of us is misunderstanding something.
I looked at some Marvell datasheet and it has a similar QoS classification pipeline to Vitesse switches. There is a port-based default priority which can be overridden by IP DSCP, VLAN PCP, or advanced QoS classification (TCAM).
Quoted 6 lines
The proposal I had was to configure the default port priority using tc matchall skbedit priority. Advanced QoS classification would then be expressed as tc-flower filters with a higher precedence than the matchall (basically the "catchall"). PCP and DSCP, I don't know if that can be expressed cleanly using tc. I think there's something in the dcb ops, but I haven't studied that too deeply.
Quoted 7 lines
In 802.1Q-2014, port-default priority is handled as APP entries matching on EtherType of 0. (See Table D-9.) Those are "default priority. For use when priority is not otherwise specified".
So DCB ops just handle these as APP entries. Dunno what DSA does. In mlxsw, we call
dcb_ieee_getapp_default_prio_mask()when the DCPset_apphook fires to find the relevant entries and get the priority bitmask.
Thanks, these are great pointers. Last time I looked at DCB ops, the dcb iproute program didn't exist, one had to use some LLDP tool IIRC, and it was a bit cumbersome and I dismissed it without even looking at all the details, I didn't notice that the port-default priority corresponds to a selector of 1 and a protocol of 0.
The point is that I'm not bent on using tc-matchall for port-based default priority, it's just that I wasn't aware of a better way. But I'll look into adding support for DCB ops for my DSA driver, sounds like a much, much better fit.
Quoted 5 lines
Now I don't understand DSA at all, but given a chip with fancy defaults, for the DCB interface in particular, it would make sense to me to have two ops. As long as there are default-prio entries, a "set default priority" op would get invoked with the highest configured default priority. When the last entry disappears, an "unset" op would be called.
I don't understand this comment, sorry. I don't know what's a "chip with fancy defaults".
Quoted 7 lines
Not sure what DSA does with ACLs, but it's not clear to me how TC-based prioritization rules coexist with full blown ACLs. I suppose the prio stuff could live on chain 0 and all actions would be skbedit prio pipe goto chain 1 or something. And goto chain 0 is forbidden, because chain 0 is special. Or maybe the prioritization stuff lives on a root qdisc (but no, we need it for ingress packets...) One way or another it looks hairy to dissect and offload accurately IMHO.
There's nothing to understand about the DSA core at all, it has no saying in how prioritization or TC rules are configured, that is left down to the hardware driver.
To make sure we use the same terminology, when you say "how TC-based prioritization rules coexist with full blown ACLs", you mean trap/drop/redirect by ACLs, right?
So the ocelot driver has a programmable, fixed pipeline of multiple ingress stages (VCAP IS1 for VLAN editing and advanced QoS classification) and egress stages (VCAP ES0 for egress VLAN rewriting). We model the entire TCAM subsystem using one chain per TCAM lookup, and force gotos from the current stage to the next. See tools/testing/selftests/drivers/net/ocelot/tc_flower_chains.sh for the intended usage model.
Now, that's all for advanced QoS classification, not for port-based default, VLAN PCP and IP DSCP. My line of thinking is that we could do the latter via dcb-app, and leave the former where it is (skbedit with tc-flower), and they'd coexist just fine, right?
Andrew Lunn
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Quoted 8 lines
Quoted 5 lines
Now I don't understand DSA at all, but given a chip with fancy defaults, for the DCB interface in particular, it would make sense to me to have two ops. As long as there are default-prio entries, a "set default priority" op would get invoked with the highest configured default priority. When the last entry disappears, an "unset" op would be called.
I don't understand this comment, sorry. I don't know what's a "chip with fancy defaults".
I think this is the point i just razed in my other reply. The hardware starts with a preconfigured QoS profile. My guess would be, a Top Of Rack defaults to all QoS features turned off, you need to turn them off as appropriate for the use case. Typically, a SOHO switch is not configured, it is just put in the corner and turned on. So having some sensible defaults for a home/office makes sense. When it is used in some other embedded use case, it probably is going to get configured for that use cases.
So 'fancy defaults' == preconfigured QoS profile in the hardware.
Andrew
Re: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
In reply to Vladimir OlteanQuoted 77 lines · Vladimir Oltean
Quoted 57 lines · Petr Machata
Quoted 47 lines · Vladimir Oltean
Hi Andrew,
Quoted 29 lines · Andrew Lunn
Quoted 9 lines · Vladimir Oltean
Quoted 7 lines · Andrew Lunn
Quoted 2 lines · Vladimir Oltean
+ int (*port_priority_set)(struct
dsa_switch*ds, int port,
+ structdsa_mall_skbedit_tc_entry*skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Quoted 6 lines
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
That does not work for mv88e6xxx. Its default setup, if i remember correctly, is it looks at the TOS bits to determine priority classes. So in its default state, it is using all the available traffic classes. It can also be configured to look at the VLAN priority, or the TCAM can set the priority class, or there is a per port default priority, which is what you are describing here. There are bits to select which of these happen on ingress, on a per port basis.
So setting the port priority to 0 means setting the priority of zero. It does not mean go back to the default prioritisation scheme.
Quoted 6 lines
I guess any switch which has a range of options for prioritisation selection will have a similar problem. It defaults to something, probably something a bit smarter than everything goes to traffic class 0.
Andrew
I was going through my old patches, and re-reading this conversation, it appears one of us is misunderstanding something.
I looked at some Marvell datasheet and it has a similar QoS classification pipeline to Vitesse switches. There is a port-based default priority which can be overridden by IP DSCP, VLAN PCP, or advanced QoS classification (TCAM).
Quoted 6 lines
The proposal I had was to configure the default port priority using tc matchall skbedit priority. Advanced QoS classification would then be expressed as tc-flower filters with a higher precedence than the matchall (basically the "catchall"). PCP and DSCP, I don't know if that can be expressed cleanly using tc. I think there's something in the dcb ops, but I haven't studied that too deeply.
Quoted 7 lines
In 802.1Q-2014, port-default priority is handled as APP entries matching on EtherType of 0. (See Table D-9.) Those are "default priority. For use when priority is not otherwise specified".
So DCB ops just handle these as APP entries. Dunno what DSA does. In mlxsw, we call
dcb_ieee_getapp_default_prio_mask()when the DCPset_apphook fires to find the relevant entries and get the priority bitmask.Thanks, these are great pointers. Last time I looked at DCB ops, the dcb iproute program didn't exist, one had to use some LLDP tool IIRC, and it was a bit cumbersome and I dismissed it without even looking at all the details, I didn't notice that the port-default priority corresponds to a selector of 1 and a protocol of 0.
The point is that I'm not bent on using tc-matchall for port-based default priority, it's just that I wasn't aware of a better way. But I'll look into adding support for DCB ops for my DSA driver, sounds like a much, much better fit.
Quoted 8 lines
Quoted 5 lines
Now I don't understand DSA at all, but given a chip with fancy defaults, for the DCB interface in particular, it would make sense to me to have two ops. As long as there are default-prio entries, a "set default priority" op would get invoked with the highest configured default priority. When the last entry disappears, an "unset" op would be called.
I don't understand this comment, sorry. I don't know what's a "chip with fancy defaults".
I'm referring here to Andrew's "I guess any switch [...] defaults to something [...] a bit smarter than everything goes to traffic class 0".
Quoted 8 lines
Quoted 7 lines
Not sure what DSA does with ACLs, but it's not clear to me how TC-based prioritization rules coexist with full blown ACLs. I suppose the prio stuff could live on chain 0 and all actions would be skbedit prio pipe goto chain 1 or something. And goto chain 0 is forbidden, because chain 0 is special. Or maybe the prioritization stuff lives on a root qdisc (but no, we need it for ingress packets...) One way or another it looks hairy to dissect and offload accurately IMHO.
Quoted 7 lines
There's nothing to understand about the DSA core at all, it has no saying in how prioritization or TC rules are configured, that is left down to the hardware driver.
To make sure we use the same terminology, when you say "how TC-based prioritization rules coexist with full blown ACLs", you mean trap/drop/redirect by ACLs, right?
Yeah. But also simple stuff, like skbedit priority, but with complex matching. Think flower match on a side chain that only gets invoked when another flower match hits.
Quoted 8 lines
So the ocelot driver has a programmable, fixed pipeline of multiple ingress stages (VCAP IS1 for VLAN editing and advanced QoS classification) and egress stages (VCAP ES0 for egress VLAN rewriting). We model the entire TCAM subsystem using one chain per TCAM lookup, and force gotos from the current stage to the next. See
tools/testing/selftests/drivers/net/ocelot/tc_flower_chains.shfor the intended usage model.
Quoted 4 lines
Now, that's all for advanced QoS classification, not for port-based default, VLAN PCP and IP DSCP. My line of thinking is that we could do the latter via dcb-app, and leave the former where it is (skbedit with tc-flower), and they'd coexist just fine, right?
That's what we do. I don't like it very much, because DCB is this odd HW-centric thing that you can't run on bridged veths. But unfortunately TC filter configuration that describes the dumb stuff and then follows up with more of the complex stuff that needs to happen _as well_, seems like it would be a mess to both dissect in the driver and use on the command line.
Maybe we need a multi-stage clsact qdisc, or something like that... ^o^
Vladimir Oltean
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
Hi Petr,
Quoted 45 lines · Petr Machata
Quoted 8 lines
Quoted 5 lines
Now I don't understand DSA at all, but given a chip with fancy defaults, for the DCB interface in particular, it would make sense to me to have two ops. As long as there are default-prio entries, a "set default priority" op would get invoked with the highest configured default priority. When the last entry disappears, an "unset" op would be called.
I don't understand this comment, sorry. I don't know what's a "chip with fancy defaults".
I'm referring here to Andrew's "I guess any switch [...] defaults to something [...] a bit smarter than everything goes to traffic class 0".
Quoted 8 lines
Quoted 7 lines
Not sure what DSA does with ACLs, but it's not clear to me how TC-based prioritization rules coexist with full blown ACLs. I suppose the prio stuff could live on chain 0 and all actions would be skbedit prio pipe goto chain 1 or something. And goto chain 0 is forbidden, because chain 0 is special. Or maybe the prioritization stuff lives on a root qdisc (but no, we need it for ingress packets...) One way or another it looks hairy to dissect and offload accurately IMHO.
Quoted 7 lines
There's nothing to understand about the DSA core at all, it has no saying in how prioritization or TC rules are configured, that is left down to the hardware driver.
To make sure we use the same terminology, when you say "how TC-based prioritization rules coexist with full blown ACLs", you mean trap/drop/redirect by ACLs, right?
Yeah. But also simple stuff, like skbedit priority, but with complex matching. Think flower match on a side chain that only gets invoked when another flower match hits.
Quoted 8 lines
So the ocelot driver has a programmable, fixed pipeline of multiple ingress stages (VCAP IS1 for VLAN editing and advanced QoS classification) and egress stages (VCAP ES0 for egress VLAN rewriting). We model the entire TCAM subsystem using one chain per TCAM lookup, and force gotos from the current stage to the next. See
tools/testing/selftests/drivers/net/ocelot/tc_flower_chains.shfor the intended usage model.Quoted 4 lines
Now, that's all for advanced QoS classification, not for port-based default, VLAN PCP and IP DSCP. My line of thinking is that we could do the latter via dcb-app, and leave the former where it is (skbedit with tc-flower), and they'd coexist just fine, right?
Quoted 8 lines
That's what we do. I don't like it very much, because DCB is this odd HW-centric thing that you can't run on bridged veths. But unfortunately TC filter configuration that describes the dumb stuff and then follows up with more of the complex stuff that needs to happen
_aswell_, seems like it would be a mess to both dissect in the driver and use on the command line.Maybe we need a multi-stage clsact qdisc, or something like that... ^o^
I see dcb_ieee_setapp() can be used to preload the Application Priority Table with information that reflects the port's configuration.
I'm just wondering - do you have any idea why the Application Priority TLV doesn't have a way to describe a mapping between VLAN PCP and priority? What can I use to also describe that?
Andrew Lunn
maintainerRe: [RFC PATCH net-next 1/2] net: dsa: allow setting port-based QoS priority using tc matchall skbedit
In reply to Vladimir OlteanQuoted 54 lines · Vladimir Oltean
Hi Andrew,
Quoted 29 lines · Andrew Lunn
Quoted 9 lines · Vladimir Oltean
Quoted 7 lines · Andrew Lunn
Quoted 2 lines · Vladimir Oltean
+ int (*port_priority_set)(struct
dsa_switch*ds, int port,
+ structdsa_mall_skbedit_tc_entry*skbedit);The fact we can turn this on/off suggests there should be a way to disable this in the hardware, when the matchall is removed. I don't see any such remove support in this patch.
Quoted 6 lines
I don't understand this comment, sorry. When the matchall filter containing the skbedit action gets removed, DSA calls the driver's .port_priority_set callback again, this time with a priority of 0. There's nothing to "remove" about a port priority. I made an assumption (which I still consider perfectly reasonable) that no port-based prioritization means that all traffic gets classified to traffic class 0.
That does not work for mv88e6xxx. Its default setup, if i remember correctly, is it looks at the TOS bits to determine priority classes. So in its default state, it is using all the available traffic classes. It can also be configured to look at the VLAN priority, or the TCAM can set the priority class, or there is a per port default priority, which is what you are describing here. There are bits to select which of these happen on ingress, on a per port basis.
So setting the port priority to 0 means setting the priority of zero. It does not mean go back to the default prioritisation scheme.
Quoted 6 lines
I guess any switch which has a range of options for prioritisation selection will have a similar problem. It defaults to something, probably something a bit smarter than everything goes to traffic class 0.
Andrew
I was going through my old patches, and re-reading this conversation, it appears one of us is misunderstanding something.
I looked at some Marvell datasheet and it has a similar QoS classification pipeline to Vitesse switches. There is a port-based default priority which can be overridden by IP DSCP, VLAN PCP, or advanced QoS classification (TCAM).
The proposal I had was to configure the default port priority using tc matchall skbedit priority. Advanced QoS classification would then be expressed as tc-flower filters with a higher precedence than the matchall (basically the "catchall"). PCP and DSCP, I don't know if that can be expressed cleanly using tc. I think there's something in the dcb ops, but I haven't studied that too deeply.
Quoted 6 lines
Anyway, I don't exactly understand your point, that an add/del is in any way better than a "set". Even for Marvell, what I'm proposing here would translate in a "set to 0" on "del" anyway. That's why this patch set is RFC. I don't know if there's a better way to express a port-based default priority than a matchall rule having the lowest precedence.
I think we have a generic problem in that the switch does not start up in a state where all QoS features are turned off. But a traditional netdev does have all QoS features off by default. You need to explicitly turn on a QoS feature on a netdev by using tc, or some other configuration mechanism.
To make the linux view of QoS features actually match what the hardware is doing, we need to preload tc with a number of rules. Your proposed 'tc matchall skbedit priority' rule might need to be already in tc because the switch might already be doing that by default, etc.
I also wonder if we need tc rules you cannot actually remove, because you cannot turn the feature off in hardware? 'tc matchall skbedit priority' is setting the default priority. If you remove the rule, the hardware is still going to apply a default priority, it is just ambiguous from tc what value it is using. It would be better if the rule was present from boot, and all you can do is change the priority, not remove the rule.
We need to consider the generic problem that the hardware comes with a preconfigured QoS profile, which we currently don't reflect in Linux. We have deployed devices which rely on that QoS profile. How do we transition to describing that preconfigured QoS profile, and allowing it to be changed?
Andrew
Vladimir Oltean
maintainerRFC PATCH 2/2 net: dsa: felix: offload port priority
In reply to Vladimir OlteanFrom: Vladimir Oltean <[email protected]>
Even though we should really share the implementation with the ocelot switchdev driver, that one needs a little bit of rework first, since its struct ocelot_port_tc only supports one tc matchall action at a time, which at the moment is used for port policers. Whereas DSA keeps a list of port-based actions in struct dsa_slave_priv::mall_tc_list, so it is much more easily extensible. It is too tempting to add the implementation for the port priority directly in Felix at the moment, which is what we do.
--- drivers/net/dsa/ocelot/felix.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/net/dsa/ocelot/felix.c +15 -0
@@ -739,6 +739,20 @@ static void felix_port_policer_del(struct dsa_switch *ds, int port)ocelot_port_policer_del (ocelot, port); } +static int felix_port_priority_set (struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit) +{ +struct ocelot *ocelot = ds->priv; + +ocelot_rmw_gix (ocelot, +ANA_PORT_QOS_CFG_QOS_DEFAULT_VAL (skbedit->priority), +ANA_PORT_QOS_CFG_QOS_DEFAULT_VAL_M , +ANA_PORT_QOS_CFG , + port); + +return 0 ; +} +static int felix_port_setup_tc (struct dsa_switch *ds,int port,enum tc_setup_type type,void *type_data)
@@ -786,6 +800,7 @@ const struct dsa_switch_ops felix_switch_ops = { .port_max_mtu = felix_get_max_mtu, .port_policer_add = felix_port_policer_add, .port_policer_del = felix_port_policer_del, + .port_priority_set = felix_port_priority_set, .cls_flower_add = felix_cls_flower_add, .cls_flower_del = felix_cls_flower_del, .cls_flower_stats = felix_cls_flower_stats,
Signature · 1 line
2.25.1
Andrew Lunn
maintainerRe: [RFC PATCH net-next 2/2] net: dsa: felix: offload port priority
Quoted 21 lines · Vladimir Oltean
From: Vladimir Oltean <[email protected]>
Even though we should really share the implementation with the ocelot switchdev driver, that one needs a little bit of rework first, since its struct
ocelot_port_tconly supports one tc matchall action at a time, which at the moment is used for port policers. Whereas DSA keeps a list of port-based actions in structdsa_slave_priv::mall_tc_list, so it is much more easily extensible. It is too tempting to add the implementation for the port priority directly in Felix at the moment, which is what we do.Signed-off-by Vladimir Oltean <[email protected]>--- drivers/net/dsa/ocelot/felix.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/net/dsa/ocelot/felix.c +0
-0
drivers/net/dsa/ocelot/felix.c static void felix_port_policer_del(struct dsa_switch *ds, int port) · line 748
Quoted 10 lines
drivers/net/dsa/ocelot/felix.c +7
-0@@ -739,6 +739,20 @@ static void felix_port_policer_del(struct dsa_switch *ds, int port)ocelot_port_policer_del (ocelot, port); } +static int felix_port_priority_set (struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit) +{ +struct ocelot *ocelot = ds->priv; + + ocelot_rmw_gix(ocelot, + ANA_PORT_QOS_CFG_QOS_DEFAULT_VAL(skbedit->priority),
No range check? Seems like -ERANGE or similar would help avoid surprises when somebody asks for an unsupported priority and it gets masked to something much lower.
Andrew
Re: [RFC PATCH net-next 2/2] net: dsa: felix: offload port priority
Quoted 25 lines · Andrew Lunn
Quoted 24 lines · Vladimir Oltean
From: Vladimir Oltean <[email protected]>
Even though we should really share the implementation with the ocelot switchdev driver, that one needs a little bit of rework first, since its struct
ocelot_port_tconly supports one tc matchall action at a time, which at the moment is used for port policers. Whereas DSA keeps a list of port-based actions in structdsa_slave_priv::mall_tc_list, so it is much more easily extensible. It is too tempting to add the implementation for the port priority directly in Felix at the moment, which is what we do.Signed-off-by Vladimir Oltean <[email protected]>--- drivers/net/dsa/ocelot/felix.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/net/dsa/ocelot/felix.c +0
-0@@ -739,6 +739,20 @@ static void felix_port_policer_del(struct dsa_switch *ds, int port)ocelot_port_policer_del (ocelot, port); }
Quoted 11 lines
drivers/net/dsa/ocelot/felix.c static void felix_port_policer_del(struct dsa_switch *ds, int port) · line 748
Quoted 7 lines
drivers/net/dsa/ocelot/felix.c +7
-0+static int felix_port_priority_set (struct dsa_switch *ds,int port, +struct dsa_mall_skbedit_tc_entry *skbedit) +{ +struct ocelot *ocelot = ds->priv; + + ocelot_rmw_gix(ocelot, + ANA_PORT_QOS_CFG_QOS_DEFAULT_VAL(skbedit->priority),No range check? Seems like
-ERANGEor similar would help avoid surprises when somebody asks for an unsupported priority and it gets masked to something much lower.
You are passing the whole dsa_mall_skbedit_tc_entry structure here, only to look up priority, would it make sense for now to pass skbedit->priority as a parameter which would be matching the function name and what it is dealing with?
Signature · 1 line
Florian
Vladimir Oltean
maintainerRe: [RFC PATCH net-next 2/2] net: dsa: felix: offload port priority
Quoted 4 lines · Florian Fainelli
You are passing the whole
dsa_mall_skbedit_tc_entrystructure here, only to look up priority, would it make sense for now to passskbedit->priorityas a parameter which would be matching the function name and what it is dealing with?
Actually I am passing a pointer to it, which should be more or less equal in size to an integer. But I can pass just the priority, sure.