tc-taprio offload for SJA1105 DSA
Subsystems (7)
Vladimir Oltean
maintainerv2 cover tc-taprio offload for SJA1105 DSA
This is the second attempt to submit the tc-taprio offload model for inclusion in the net tree. The sja1105 switch driver will provide the first implementation of the offload. Only the bare minimum is added:
- The offload model and a DSA pass-through
- The hardware implementation
- The interaction with the netdev queues in the tagger code
- Documentation
What has been removed from the first attempt is support for PTP-as-clocksource in sja1105. This will be added as soon as the offload model is settled.
Vinicius Costa Gomes (1):
taprio: Add support for hardware offloading
Vladimir Oltean (6):
net: dsa: Pass ndo_setup_tc slave callback to drivers
net: dsa: sja1105: Add static config tables for scheduling
net: dsa: sja1105: Advertise the 8 TX queues
net: dsa: sja1105: Make HOSTPRIO a kernel config
net: dsa: sja1105: Configure the Time-Aware Scheduler via tc-taprio
offload
docs: net: dsa: sja1105: Add info about the time-aware scheduler
Documentation/networking/dsa/sja1105.rst | 90 ++++ drivers/net/dsa/sja1105/Kconfig | 17 + drivers/net/dsa/sja1105/Makefile | 4 + drivers/net/dsa/sja1105/sja1105.h | 6 + .../net/dsa/sja1105/sja1105_dynamic_config.c | 8 + drivers/net/dsa/sja1105/sja1105_main.c | 28 +- .../net/dsa/sja1105/sja1105_static_config.c | 167 +++++++ .../net/dsa/sja1105/sja1105_static_config.h | 48 +- drivers/net/dsa/sja1105/sja1105_tas.c | 419 ++++++++++++++++++ drivers/net/dsa/sja1105/sja1105_tas.h | 44 ++ include/linux/netdevice.h | 1 + include/net/dsa.h | 2 + include/net/pkt_sched.h | 23 + include/uapi/linux/pkt_sched.h | 3 +- net/dsa/slave.c | 12 +- net/dsa/tag_sja1105.c | 3 +- net/sched/sch_taprio.c | 409 +++++++++++++++-- 17 files changed, 1231 insertions(+), 53 deletions(-) create mode 100644 drivers/net/dsa/sja1105/sja1105_tas.c create mode 100644 drivers/net/dsa/sja1105/sja1105_tas.h
Signature · 106 lines
For those who want to follow along with the hardware implementation, the
manual is here:
https://www.nxp.com/docs/en/user-guide/UM10944.pdf
Notable changes in v2:
- Made the series independent from PTP (which is temporarily removed)
- Changed the meaning of the gate_mask - it is now acting on traffic
classes even in the view exposed by taprio to drivers.
- Removed the next_sched hrtimer.
- Summarized one of the responses given to Vinicius into a new
documentation section.
The first version of the net-next patch series can be found here:
https://www.spinics.net/lists/netdev/msg597214.html
Changes in the first version of the net-next series compared to RFC v2:
- Made "flags 1" and "flags 2" mutually exclusive in the taprio qdisc
- Moved taprio_enable_offload and taprio_disable_offload out of atomic
context - spin_lock_bh(qdisc_lock(sch)). This allows drivers that
implement the ndo_setup_tc to sleep and for taprio memory to be
allocated with GFP_KERNEL. The only thing that was kept under the
spinlock is the assignment of the q->dequeue and q->peek pointers.
- Finally making proper use of own API - added a taprio_alloc helper to
avoid passing stack memory to drivers.
The second version of the RFC is at:
https://www.spinics.net/lists/netdev/msg596663.html
Changes in v2 of the RFC since v1:
- Adapted the taprio offload patch to work by specifying "flags 2" to
the iproute2-next tc. At the moment I don't clearly understand whether
the full offload and the txtime assist ("flags 1") are mutually
exclusive or not (i.e. whether a "flags 3" mode should be rejected,
which it currently isn't).
- Added reference counting to the taprio offload structure. Maybe the
function names and placement could have been better though. As for the
other complaint (cycle time calculation) it got fixed in the taprio
parser in the meantime.
- Converted sja1105 to use the hardware PTP registers, and save/restore
the PTP time across resets.
- Made the DSA callback for ndo_setup_tc a bit more generic, but I don't
know whether it fulfills expectations. Drivers still can't do blocking
operations in its execution context.
- Added a state machine for starting/stopping the scheduler based on the
last command run on the PTP clock.
The first RFC from July can be seen at:
https://lists.openwall.net/netdev/2019/07/07/81
Original cover letter:
Using Vinicius Costa Gomes' configuration interface for 802.1Qbv (later
resent by Voon Weifeng for the stmmac driver), I am submitting for
review a draft implementation of this offload for a DSA switch.
I don't want to insist too much on the hardware specifics of SJA1105
which isn't otherwise very compliant to the IEEE spec.
In order to be able to test with Vedang Patel's iproute2 patch for
taprio offload (https://www.spinics.net/lists/netdev/msg573072.html)
I had to actually revert the txtime-assist branch as it had changed the
iproute2 interface.
In terms of impact for DSA drivers, I would like to point out that:
- Maybe somebody should pre-populate qopt->cycle_time in case the user
does not provide one. Otherwise each driver needs to iterate over the
GCL once, just to set the cycle time (right now stmmac does as well).
- Configuring the switch over SPI cannot apparently be done from this
ndo_setup_tc callback because it runs in atomic context. I also have
some downstream patches to offload tc clsact matchall with mirred
action, but in that case it looks like the atomic context restriction
does not apply.
- I had to copy the struct tc_taprio_qopt_offload to driver private
memory because a static config needs to be constructed every time a
change takes place, and there are up to 4 switch ports that may take a
TAS configuration. I have created a private
tc_taprio_qopt_offload_copy() helper for this - I don't know whether
it's of any help in the general case.
There is more to be done however. The TAS needs to be integrated with
the PTP driver. This is because with a PTP clock source, the base time
is written dynamically to the PTPSCHTM (PTP schedule time) register and
must be a time in the future. Then the "real" base time of each port's
TAS config can be offset by at most ~50 ms (the DELTA field from the
Schedule Entry Points Table) relative to PTPSCHTM.
Because base times in the past are completely ignored by this hardware,
we need to decide if it's ok behaviorally for a driver to "roll" a past
base time into the immediate future by incrementally adding the cycle
time (so the phase doesn't change). If it is, then decide by how long in
the future it is ok to do so. Or alternatively, is it preferable if the
driver errors out if the user-supplied base time is in the past and the
hardware doesn't like it? But even then, there might be fringe cases
when the base time becomes a past PTP time right as the driver tries to
apply the config.
Also applying a tc-taprio offload to a second SJA1105 switch port will
inevitably need to roll the first port's (now past) base time into an
equivalent future time.
All of this is going to be complicated even further by the fact that
resetting the switch (to apply the tc-taprio offload) makes it reset its
PTP time.
2.17.1Vladimir Oltean
maintainerPATCH v2 1/7 taprio: Add support for hardware offloading
From: Vinicius Costa Gomes <[email protected]>
This allows taprio to offload the schedule enforcement to capable network cards, resulting in more precise windows and less CPU usage.
The gate mask acts on traffic classes (groups of queues of same priority), as specified in IEEE 802.1Q-2018, and following the existing taprio and mqprio semantics.
It is up to the driver to perform conversion between tc and individual netdev queues if for some reason it needs to make that distinction.
Full offload is requested from the network interface by specifying "flags 2" in the tc qdisc creation command, which in turn corresponds to the TCA_TAPRIO_ATTR_FLAG_FULL_OFFLOAD bit.
The important detail here is the clockid which is implicitly /dev/ptpN for full offload, and hence not configurable.
A reference counting API is added to support the use case where Ethernet drivers need to keep the taprio offload structure locally (i.e. they are a multi-port switch driver, and configuring a port depends on the settings of other ports as well). The refcount_t variable is kept in a private structure (__tc_taprio_qopt_offload) and not exposed to drivers.
In the future, the private structure might also be expanded with a backpointer to taprio_sched *q, to implement the notification system described in the patch (of when admin became oper, or an error occurred, etc, so the offload can be monitored with 'tc qdisc show').
---
Changes since v1:
- Turned the next_sched hrtimer function into a simple taprio_offload_config_changed function called synchronously (for now) from taprio_enable_offload. But the idea is that the driver may have a lot more means to figure out when the admin schedule is no longer pending (perhaps even an interrupt), so leave an open window for implementing a notification system from the driver.
- Made it an error to specify 'clockid' with full offload.
- Created a wrapper __tc_taprio_qopt_offload structure which holds the refcount_t (for now) and maybe a backpointer to the qdisc_priv in the future.
- Renamed taprio_get and taprio_free to taprio_offload_get and taprio_offload_free. Renamed the "taprio" variable to "offload".
- Moved the reference counting helper implementations to sch_taprio.c.
- Removed the tc_mask_to_queue_mask manipulation done to the gate_mask before passing it on to drivers. Instead of netdev queue gates, they now see a mask of traffic class gates, which:
- They need to care about anyway, if they have a multi-queue device and they need to configure the queue-to-tc hardware mapping.
- Makes no difference to them if the hardware makes no distinction between queue and traffic class (there is only one egress queue per tc, having a fixed priority). The sja1105 hw is in this situation.
Changes since RFC:
- Made the combination of FULL_OFFLOAD and TXTIME_ASSIST invalid.
- Made ndo_setup_tc be called from sleepable context.
- Added a taprio_alloc helper to avoid passing stack memory to drivers.
- Made taprio_disable_offload take the extack as well.
- Conditioned the setup of the software (and txtime-assisted) implementation of taprio on there not being a full offload in place.
- Fixed a lockdep-related compilation bug.
include/linux/netdevice.h | 1 + include/net/pkt_sched.h | 23 ++ include/uapi/linux/pkt_sched.h | 3 +- net/sched/sch_taprio.c | 409 +++++++++++++++++++++++++++++---- 4 files changed, 392 insertions(+), 44 deletions(-)
include/linux/netdevice.h +1 -0
@@ -847,6 +847,7 @@ enum tc_setup_type {TC_SETUP_QDISC_ETF ,TC_SETUP_ROOT_QDISC ,TC_SETUP_QDISC_GRED , + TC_SETUP_QDISC_TAPRIO, };/* These structures hold the attributes of bpf state that are being passed
include/net/pkt_sched.h +23 -0
@@ -161,4 +161,27 @@ struct tc_etf_qopt_offload { s32 queue; }; +struct tc_taprio_sched_entry { +u8 command;/* TC_TAPRIO_CMD_* */ + +/* The gate_mask in the offloading side refers to traffic classes */ +u32 gate_mask; +u32 interval; +}; + +struct tc_taprio_qopt_offload { +u8 enable; +ktime_t base_time; +u64 cycle_time; +u64 cycle_time_extension; + +size_t num_entries; +struct tc_taprio_sched_entry entries[0 ]; +}; + +/* Reference counting */ +struct tc_taprio_qopt_offload *taprio_offload_get (struct tc_taprio_qopt_offload + *offload); +void taprio_offload_free (struct tc_taprio_qopt_offload *offload); +#endif
include/uapi/linux/pkt_sched.h +2 -1
@@ -1160,7 +1160,8 @@ enum {* [TCA_TAPRIO_ATTR_SCHED_ENTRY_INTERVAL] */ -#define TCA_TAPRIO_ATTR_FLAG_TXTIME_ASSIST 0x1 +#define TCA_TAPRIO_ATTR_FLAG_TXTIME_ASSIST BIT(0) +#define TCA_TAPRIO_ATTR_FLAG_FULL_OFFLOAD BIT(1)enum {TCA_TAPRIO_ATTR_UNSPEC ,
net/sched/sch_taprio.c +366 -43
521 lines not on this page · raw message
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH v2 2/7 net: dsa: Pass ndo_setup_tc slave callback to drivers
In reply to Vladimir OlteanDSA currently handles shared block filters (for the classifier-action qdisc) in the core due to what I believe are simply pragmatic reasons - hiding the complexity from drivers and offerring a simple API for port mirroring.
Extend the dsa_slave_setup_tc function by passing all other qdisc offloads to the driver layer, where the driver may choose what it implements and how. DSA is simply a pass-through in this case.
---
Changes since v1:
- Added Kurt Kanzenbach's Acked-by.
Changes since RFC:
- Removed the unused declaration of struct tc_taprio_qopt_offload.
include/net/dsa.h | 2 ++ net/dsa/slave.c | 12 ++++++++---- 2 files changed, 10 insertions(+), 4 deletions(-)
include/net/dsa.h +2 -0
@@ -515,6 +515,8 @@ struct dsa_switch_ops { bool ingress);void (*port_mirror_del)(struct dsa_switch *ds,int port,struct dsa_mall_mirror_tc_entry *mirror); +int (*port_setup_tc)(struct dsa_switch *ds,int port, +enum tc_setup_type type,void *type_data);/* * Cross-chip operations
net/dsa/slave.c +8 -4
@@ -1035,12 +1035,16 @@ static int dsa_slave_setup_tc_block(struct net_device *dev,static int dsa_slave_setup_tc (struct net_device *dev,enum tc_setup_type type,void *type_data) { -switch (type) { -case TC_SETUP_BLOCK : +struct dsa_port *dp =dsa_slave_to_port (dev); +struct dsa_switch *ds = dp->ds; + +if (type ==TC_SETUP_BLOCK )return dsa_slave_setup_tc_block (dev, type_data); -default : + +if (!ds->ops->port_setup_tc)return -EOPNOTSUPP ; - } + +return ds->ops->port_setup_tc (ds, dp->index, type, type_data); } static void dsa_slave_get_stats64(struct net_device *dev,
Signature · 1 line
2.17.1
Re: [PATCH v2 net-next 2/7] net: dsa: Pass ndo_setup_tc slave callback to drivers
Quoted 11 lines · Vladimir Oltean
DSA currently handles shared block filters (for the classifier-action qdisc) in the core due to what I believe are simply pragmatic reasons - hiding the complexity from drivers and offerring a simple API for port mirroring.
Extend the
dsa_slave_setup_tcfunction by passing all other qdisc offloads to the driver layer, where the driver may choose what it implements and how. DSA is simply a pass-through in this case.Signed-off-by Vladimir Oltean <[email protected]>Acked-by Kurt Kanzenbach <[email protected]>
Signature · 1 line
Florian
Re: [PATCH v2 net-next 2/7] net: dsa: Pass ndo_setup_tc slave callback to drivers
In reply to Vladimir OlteanHi Vladimir,
Yes fixes my request on the initial RFC. Sorry for the delayed response.
Quoted 63 lines · Vladimir Oltean
DSA currently handles shared block filters (for the classifier-action qdisc) in the core due to what I believe are simply pragmatic reasons - hiding the complexity from drivers and offerring a simple API for port mirroring.
Extend the
dsa_slave_setup_tcfunction by passing all other qdisc offloads to the driver layer, where the driver may choose what it implements and how. DSA is simply a pass-through in this case.Signed-off-by Vladimir Oltean <[email protected]>Acked-by Kurt Kanzenbach <[email protected]>---
Changes since v1:
- Added Kurt Kanzenbach's Acked-by.Changes since RFC:
- Removed the unused declaration of structtc_taprio_qopt_offload.include/net/dsa.h | 2 ++ net/dsa/slave.c | 12 ++++++++---- 2 files changed, 10 insertions(+), 4 deletions(-)include/net/dsa.h +2
-0@@ -515,6 +515,8 @@ struct dsa_switch_ops { bool ingress);void (*port_mirror_del)(struct dsa_switch *ds,int port,struct dsa_mall_mirror_tc_entry *mirror); +int (*port_setup_tc)(struct dsa_switch *ds,int port, +enum tc_setup_type type,void *type_data);/* * Cross-chip operations net/dsa/slave.c +8
-4@@ -1035,12 +1035,16 @@ static int dsa_slave_setup_tc_block(struct net_device *dev,static int dsa_slave_setup_tc (struct net_device *dev,enum tc_setup_type type,void *type_data) { -switch (type) { -case TC_SETUP_BLOCK : +struct dsa_port *dp =dsa_slave_to_port (dev); +struct dsa_switch *ds = dp->ds; + +if (type ==TC_SETUP_BLOCK )return dsa_slave_setup_tc_block (dev, type_data); -default : + +if (!ds->ops->port_setup_tc)return -EOPNOTSUPP ; - } + +return ds->ops->port_setup_tc (ds, dp->index, type, type_data); } static void dsa_slave_get_stats64(struct net_device *dev,Signature · 1 line
2.17.1
Vladimir Oltean
maintainerRe: [PATCH v2 net-next 2/7] net: dsa: Pass ndo_setup_tc slave callback to drivers
Hi Ilias,
Quoted 65 lines · Ilias Apalodimas
Hi Vladimir,
Yes fixes my request on the initial RFC. Sorry for the delayed response.
Quoted 59 lines · Vladimir Oltean
DSA currently handles shared block filters (for the classifier-action qdisc) in the core due to what I believe are simply pragmatic reasons - hiding the complexity from drivers and offerring a simple API for port mirroring.
Extend the
dsa_slave_setup_tcfunction by passing all other qdisc offloads to the driver layer, where the driver may choose what it implements and how. DSA is simply a pass-through in this case.Signed-off-by Vladimir Oltean <[email protected]>Acked-by Kurt Kanzenbach <[email protected]>---
Changes since v1:
- Added Kurt Kanzenbach's Acked-by.Changes since RFC:
- Removed the unused declaration of structtc_taprio_qopt_offload.include/net/dsa.h | 2 ++ net/dsa/slave.c | 12 ++++++++---- 2 files changed, 10 insertions(+), 4 deletions(-)include/net/dsa.h +2
-0@@ -515,6 +515,8 @@ struct dsa_switch_ops { bool ingress);void (*port_mirror_del)(struct dsa_switch *ds,int port,struct dsa_mall_mirror_tc_entry *mirror); +int (*port_setup_tc)(struct dsa_switch *ds,int port, +enum tc_setup_type type,void *type_data);/* * Cross-chip operations net/dsa/slave.c +8
-4@@ -1035,12 +1035,16 @@ static int dsa_slave_setup_tc_block(struct net_device *dev,static int dsa_slave_setup_tc (struct net_device *dev,enum tc_setup_type type,void *type_data) { -switch (type) { -case TC_SETUP_BLOCK : +struct dsa_port *dp =dsa_slave_to_port (dev); +struct dsa_switch *ds = dp->ds; + +if (type ==TC_SETUP_BLOCK )return dsa_slave_setup_tc_block (dev, type_data); -default : + +if (!ds->ops->port_setup_tc)return -EOPNOTSUPP ; - } + +return ds->ops->port_setup_tc (ds, dp->index, type, type_data); }
Quoted 6 lines
Quoted 4 lines
net/dsa/slave.c +0
-1static void dsa_slave_get_stats64(struct net_device *dev, --2.17.1
Acked-by Ilias Apalodimas <[email protected]>
Thanks. Could you add this Acked-by to the v4 email, just so we keep the discussion on the latest submitted version?
-Vladimir
Vladimir Oltean
maintainerPATCH v2 3/7 net: dsa: sja1105: Add static config tables for scheduling
In reply to Vladimir OlteanIn order to support tc-taprio offload, the TTEthernet egress scheduling core registers must be made visible through the static interface.
---
Changes since v1:
- None.
Changes since RFC:
- None.
.../net/dsa/sja1105/sja1105_dynamic_config.c | 8 + .../net/dsa/sja1105/sja1105_static_config.c | 167 ++++++++++++++++++ .../net/dsa/sja1105/sja1105_static_config.h | 48 ++++- 3 files changed, 222 insertions(+), 1 deletion(-)
drivers/net/dsa/sja1105/sja1105_dynamic_config.c +8 -0
@@ -488,6 +488,8 @@ sja1105et_general_params_entry_packing(void *buf, void *entry_ptr,/* SJA1105E/T: First generation */ struct sja1105_dynamic_table_ops sja1105et_dyn_ops[BLK_IDX_MAX_DYN ] = { + [BLK_IDX_SCHEDULE ] = {0 }, + [BLK_IDX_SCHEDULE_ENTRY_POINTS ] = {0 }, [BLK_IDX_L2_LOOKUP ] = { .entry_packing = sja1105et_dyn_l2_lookup_entry_packing, .cmd_packing = sja1105et_l2_lookup_cmd_packing,
@@ -529,6 +531,8 @@ struct sja1105_dynamic_table_ops sja1105et_dyn_ops[BLK_IDX_MAX_DYN] = { .packed_size = SJA1105ET_SIZE_MAC_CONFIG_DYN_CMD, .addr = 0x36, }, + [BLK_IDX_SCHEDULE_PARAMS] = {0}, + [BLK_IDX_SCHEDULE_ENTRY_POINTS_PARAMS] = {0}, [BLK_IDX_L2_LOOKUP_PARAMS] = { .entry_packing = sja1105et_l2_lookup_params_entry_packing, .cmd_packing = sja1105et_l2_lookup_params_cmd_packing,
@@ -552,6 +556,8 @@ struct sja1105_dynamic_table_ops sja1105et_dyn_ops[BLK_IDX_MAX_DYN] = {/* SJA1105P/Q/R/S: Second generation */ struct sja1105_dynamic_table_ops sja1105pqrs_dyn_ops[BLK_IDX_MAX_DYN ] = { + [BLK_IDX_SCHEDULE ] = {0 }, + [BLK_IDX_SCHEDULE_ENTRY_POINTS ] = {0 }, [BLK_IDX_L2_LOOKUP ] = { .entry_packing = sja1105pqrs_dyn_l2_lookup_entry_packing, .cmd_packing = sja1105pqrs_l2_lookup_cmd_packing,
@@ -593,6 +599,8 @@ struct sja1105_dynamic_table_ops sja1105pqrs_dyn_ops[BLK_IDX_MAX_DYN] = { .packed_size = SJA1105PQRS_SIZE_MAC_CONFIG_DYN_CMD, .addr = 0x4B, }, + [BLK_IDX_SCHEDULE_PARAMS] = {0}, + [BLK_IDX_SCHEDULE_ENTRY_POINTS_PARAMS] = {0}, [BLK_IDX_L2_LOOKUP_PARAMS] = { .entry_packing = sja1105et_l2_lookup_params_entry_packing, .cmd_packing = sja1105et_l2_lookup_params_cmd_packing,
drivers/net/dsa/sja1105/sja1105_static_config.c +167 -0
268 lines not on this page · raw message
drivers/net/dsa/sja1105/sja1105_static_config.h +47 -1
109 lines not on this page · raw message
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH v2 4/7 net: dsa: sja1105: Advertise the 8 TX queues
In reply to Vladimir OlteanThis is a preparation patch for the tc-taprio offload (and potentially for other future offloads such as tc-mqprio).
Instead of looking directly at skb->priority during xmit, let's get the netdev queue and the queue-to-traffic-class mapping, and put the resulting traffic class into the dsa_8021q PCP field. The switch is configured with a 1-to-1 PCP-to-ingress-queue-to-egress-queue mapping (see vlan_pmap in sja1105_main.c), so the effect is that we can inject into a front-panel's egress traffic class through VLAN tagging from Linux, completely transparently.
Unfortunately the switch doesn't look at the VLAN PCP in the case of management traffic to/from the CPU (link-local frames at 01-80-C2-xx-xx-xx or 01-1B-19-xx-xx-xx) so we can't alter the transmission queue of this type of traffic on a frame-by-frame basis. It is only selected through the "hostprio" setting which ATM is harcoded in the driver to 7.
---
Changes since v1:
- None, but the use of netdev_txq_to_tc is now finally correct after adjusting the gate_mask meaning in the taprio offload structure.
Changes since RFC:
- None.
drivers/net/dsa/sja1105/sja1105_main.c | 7 ++++++- net/dsa/tag_sja1105.c | 3 ++- 2 files changed, 8 insertions(+), 2 deletions(-)
drivers/net/dsa/sja1105/sja1105_main.c +6 -1
@@ -384,7 +384,9 @@ static int sja1105_init_general_params(struct sja1105_private *priv)/* Disallow dynamic changing of the mirror port */ .mirr_ptacu = 0, .switchid = priv->ds->index, -/* Priority queue for link-local frames trapped to CPU */ +/* Priority queue for link-local management frames +* (both ingress to and egress from CPU - PTP, STP etc) +*/ .hostprio = 7, .mac_fltres1 = SJA1105_LINKLOCAL_FILTER_A, .mac_flt1 = SJA1105_LINKLOCAL_FILTER_A_MASK,
@@ -1711,6 +1713,9 @@ static int sja1105_setup(struct dsa_switch *ds)*/ ds->vlan_filtering_is_global = true; +/* Advertise the 8 egress queues */ + ds->num_tx_queues =SJA1105_NUM_TC ; +/* The DSA/switchdev model brings up switch ports in standalone mode by * default, and that means vlan_filtering is 0 since they're not under * a bridge, so it's safe to set up switch tagging at this time.
net/dsa/tag_sja1105.c +2 -1
@@ -89,7 +89,8 @@ static struct sk_buff *sja1105_xmit(struct sk_buff *skb,struct dsa_port *dp =dsa_slave_to_port (netdev);struct dsa_switch *ds = dp->ds;u16 tx_vid =dsa_8021q_tx_vid (ds, dp->index); -u8 pcp = skb->priority; +u16 queue_mapping =skb_get_queue_mapping (skb); +u8 pcp =netdev_txq_to_tc (netdev, queue_mapping);/* Transmitting management traffic does not rely upon switch tagging, * but instead SPI-installed management routes. Part 2 of this
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH v2 5/7 net: dsa: sja1105: Make HOSTPRIO a kernel config
In reply to Vladimir OlteanUnfortunately with this hardware, there is no way to transmit in-band QoS hints with management frames (i.e. VLAN PCP is ignored). The traffic class for these is fixed in the static config (which in turn requires a reset to change).
With the new ability to add time gates for individual traffic classes, there is a real danger that the user might unknowingly turn off the traffic class for PTP, BPDUs, LLDP etc.
So we need to manage this situation the best we can. There isn't any knob in Linux for this, and changing it at runtime probably isn't worth it either. So just make the setting loud enough by promoting it to a Kconfig, which the user can customize to their particular setup.
---
Changes since v1:
- None.
Changes since RFC:
- None.
drivers/net/dsa/sja1105/Kconfig | 9 +++++++++ drivers/net/dsa/sja1105/sja1105_main.c | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-)
drivers/net/dsa/sja1105/Kconfig +9 -0
@@ -17,6 +17,15 @@ tristate "NXP SJA1105 Ethernet switch family support" - SJA1105R (Gen. 2, SGMII, No TT-Ethernet) - SJA1105S (Gen. 2, SGMII, TT-Ethernet) +config NET_DSA_SJA1105_HOSTPRIO + int "Traffic class for management traffic" + range 0 7 + default 7 + depends on NET_DSA_SJA1105 + help + Configure the traffic class which will be used for management + (link-local) traffic sent and received over switch ports. + config NET_DSA_SJA1105_PTP bool "Support for the PTP clock on the NXP SJA1105 Ethernet switch" depends on NET_DSA_SJA1105
drivers/net/dsa/sja1105/sja1105_main.c +1 -1
@@ -387,7 +387,7 @@ static int sja1105_init_general_params(struct sja1105_private *priv)/* Priority queue for link-local management frames * (both ingress to and egress from CPU - PTP, STP etc) */ - .hostprio = 7, + .hostprio = CONFIG_NET_DSA_SJA1105_HOSTPRIO, .mac_fltres1 = SJA1105_LINKLOCAL_FILTER_A, .mac_flt1 = SJA1105_LINKLOCAL_FILTER_A_MASK, .incl_srcpt1 = false,
Signature · 1 line
2.17.1
Vladimir Oltean
maintainerPATCH v2 6/7 net: dsa: sja1105: Configure the Time-Aware Scheduler via tc-taprio offload
In reply to Vladimir OlteanThis qdisc offload is the closest thing to what the SJA1105 supports in hardware for time-based egress shaping. The switch core really is built around SAE AS6802/TTEthernet (a TTTech standard) but can be made to operate similarly to IEEE 802.1Qbv with some constraints:
- The gate control list is a global list for all ports. There are 8 execution threads that iterate through this global list in parallel. I don't know why 8, there are only 4 front-panel ports.
- Care must be taken by the user to make sure that two execution threads never get to execute a GCL entry simultaneously. I created a O(n^4) checker for this hardware limitation, prior to accepting a taprio offload configuration as valid.
- The spec says that if a GCL entry's interval is shorter than the frame length, you shouldn't send it (and end up in head-of-line blocking). Well, this switch does anyway.
- The switch has no concept of ADMIN and OPER configurations. Because it's so simple, the TAS settings are loaded through the static config tables interface, so there isn't even place for any discussion about 'graceful switchover between ADMIN and OPER'. You just reset the switch and upload a new OPER config.
- The switch accepts multiple time sources for the gate events. Right now I am using the standalone clock source as opposed to PTP. So the base time parameter doesn't really do much. Support for the PTP clock source will be added in a future series.
---
Changes since v1:
- Adapted to the naming convention changes in 01/07 (taprio_get -> taprio_offload_get, tas_config -> offload, etc).
Changes since RFC:
- Removed the sja1105_tas_config_work workqueue.
- Allocating memory with GFP_KERNEL.
- Made the ASCII art drawing fit in < 80 characters.
- Made most of the time-holding variables s64 instead of u64 (for fear of them not holding the result of signed arithmetics properly).
drivers/net/dsa/sja1105/Kconfig | 8 + drivers/net/dsa/sja1105/Makefile | 4 + drivers/net/dsa/sja1105/sja1105.h | 6 + drivers/net/dsa/sja1105/sja1105_main.c | 19 +- drivers/net/dsa/sja1105/sja1105_tas.c | 419 +++++++++++++++++++++++++ drivers/net/dsa/sja1105/sja1105_tas.h | 44 +++ 6 files changed, 499 insertions(+), 1 deletion(-) create mode 100644 drivers/net/dsa/sja1105/sja1105_tas.c create mode 100644 drivers/net/dsa/sja1105/sja1105_tas.h
drivers/net/dsa/sja1105/Kconfig +8 -0
@@ -32,3 +32,11 @@ config NET_DSA_SJA1105_PTP help This enables support for timestamping and PTP clock manipulations in the SJA1105 DSA driver. + +config NET_DSA_SJA1105_TAS + bool "Support for the Time-Aware Scheduler on NXP SJA1105" + depends on NET_DSA_SJA1105 + help + This enables support for the TTEthernet-based egress scheduling + engine in the SJA1105 DSA driver, which is controlled using a + hardware offload of the tc-tqprio qdisc.
drivers/net/dsa/sja1105/Makefile +4 -0
@@ -12,3 +12,7 @@ sja1105-objs := \ ifdef CONFIG_NET_DSA_SJA1105_PTP sja1105-objs += sja1105_ptp.o endif + +ifdef CONFIG_NET_DSA_SJA1105_TAS +sja1105-objs += sja1105_tas.o +endif
drivers/net/dsa/sja1105/sja1105.h +6 -0
@@ -20,6 +20,8 @@*/ #define SJA1105_AGEING_TIME_MS (ms) ((ms) / 10) +#include "sja1105_tas.h" +/* Keeps the different addresses between E/T and P/Q/R/S */ struct sja1105_regs {u64 device_id;
@@ -104,6 +106,7 @@ struct sja1105_private {*/ struct mutex mgmt_lock;struct sja1105_tagger_data tagger_data; +struct sja1105_tas_data tas_data; };#include "sja1105_dynamic_config.h"
@@ -120,6 +123,9 @@ typedef enum {SPI_WRITE =1 , } sja1105_spi_rw_mode_t; +/* From sja1105_main.c */ +int sja1105_static_config_reload (struct sja1105_private *priv); +/* From sja1105_spi.c */ int sja1105_spi_send_packed_buf(const struct sja1105_private *priv, sja1105_spi_rw_mode_t rw, u64 reg_addr,
drivers/net/dsa/sja1105/sja1105_main.c +18 -1
@@ -22,6 +22,7 @@#include <linux/if_ether.h> #include <linux/dsa/8021q.h> #include "sja1105.h" +#include "sja1105_tas.h" static void sja1105_hw_reset (struct gpio_desc *gpio,unsigned int pulse_len,unsigned int startup_delay)
@@ -1382,7 +1383,7 @@ static void sja1105_bridge_leave(struct dsa_switch *ds, int port,* modify at runtime (currently only MAC) and restore them after uploading, * such that this operation is relatively seamless. */ -static int sja1105_static_config_reload (struct sja1105_private *priv) +int sja1105_static_config_reload (struct sja1105_private *priv) {struct sja1105_mac_config_entry *mac;int speed_mbps[SJA1105_NUM_PORTS ];
@@ -1727,6 +1728,7 @@ static void sja1105_teardown(struct dsa_switch *ds) {struct sja1105_private *priv = ds->priv; +sja1105_tas_teardown (priv);cancel_work_sync (&priv->tagger_data.rxtstamp_work);skb_queue_purge (&priv->tagger_data.skb_rxtstamp_queue);sja1105_ptp_clock_unregister (priv);
@@ -2056,6 +2058,18 @@ static bool sja1105_port_txtstamp(struct dsa_switch *ds, int port,return true; } +static int sja1105_port_setup_tc (struct dsa_switch *ds,int port, +enum tc_setup_type type, +void *type_data) +{ +switch (type) { +case TC_SETUP_QDISC_TAPRIO : +return sja1105_setup_tc_taprio (ds, port, type_data); +default : +return -EOPNOTSUPP ; + } +} +static const struct dsa_switch_ops sja1105_switch_ops = { .get_tag_protocol = sja1105_get_tag_protocol, .setup = sja1105_setup,
@@ -2088,6 +2102,7 @@ static const struct dsa_switch_ops sja1105_switch_ops = { .port_hwtstamp_set = sja1105_hwtstamp_set, .port_rxtstamp = sja1105_port_rxtstamp, .port_txtstamp = sja1105_port_txtstamp, + .port_setup_tc = sja1105_port_setup_tc, };static int sja1105_check_device_id (struct sja1105_private *priv)
@@ -2197,6 +2212,8 @@ static int sja1105_probe(struct spi_device *spi) }mutex_init (&priv->mgmt_lock); +sja1105_tas_setup (priv); +return dsa_register_switch (priv->ds); }
drivers/net/dsa/sja1105/sja1105_tas.c new file +419 -0
419 lines not on this page · raw message
drivers/net/dsa/sja1105/sja1105_tas.h new file +44 -0
44 lines not on this page · raw message
Signature · 1 line
2.17.1
kbuild test robot
build botRe: [PATCH v2 net-next 6/7] net: dsa: sja1105: Configure the Time-Aware Scheduler via tc-taprio offload
I love your patch! Yet something to improve:
Hi Vladimir,
I love your patch! Yet something to improve:
[auto build test ERROR on net-next/master]
url: https://github.com/0day-ci/linux/commits/Vladimir-Oltean/tc-taprio-offload-for-SJA1105-DSA/20190914-154650
config: i386-randconfig-b001-201936 (attached as .config)
compiler: gcc-7 (Debian 7.4.0-11) 7.4.0 reproduce:
# save the attached .config to linux build tree
make ARCH=i386 If you fix the issue, kindly add following tag
All error/warnings (new ones prefixed by >>):
In file included from drivers/net/dsa/sja1105/sja1105.h:23:0,
from drivers/net/dsa/sja1105/sja1105_spi.c:8:
>> drivers/net/dsa/sja1105/sja1105_tas.h:38:45: warning: 'struct sja1105_private' declared inside parameter list will not be visible outside of this definition or declaration
static inline void sja1105_tas_setup(struct sja1105_private *priv) { }
^~~~~~~~~~~~~~~
drivers/net/dsa/sja1105/sja1105_tas.h:40:48: warning: 'struct sja1105_private' declared inside parameter list will not be visible outside of this definition or declaration
static inline void sja1105_tas_teardown(struct sja1105_private *priv) { }
^~~~~~~~~~~~~~~
--
In file included from drivers/net/dsa/sja1105/sja1105.h:23:0,
from drivers/net/dsa/sja1105/sja1105_main.c:24:
>> drivers/net/dsa/sja1105/sja1105_tas.h:38:45: warning: 'struct sja1105_private' declared inside parameter list will not be visible outside of this definition or declaration
static inline void sja1105_tas_setup(struct sja1105_private *priv) { }
^~~~~~~~~~~~~~~
drivers/net/dsa/sja1105/sja1105_tas.h:40:48: warning: 'struct sja1105_private' declared inside parameter list will not be visible outside of this definition or declaration
static inline void sja1105_tas_teardown(struct sja1105_private *priv) { }
^~~~~~~~~~~~~~~
drivers/net/dsa/sja1105/sja1105_main.c: In function 'sja1105_teardown':
>> drivers/net/dsa/sja1105/sja1105_main.c:1731:23: error: passing argument 1 of 'sja1105_tas_teardown' from incompatible pointer type [-Werror=incompatible-pointer-types]
sja1105_tas_teardown(priv);
^~~~
In file included from drivers/net/dsa/sja1105/sja1105.h:23:0,
from drivers/net/dsa/sja1105/sja1105_main.c:24:
drivers/net/dsa/sja1105/sja1105_tas.h:40:20: note: expected 'struct sja1105_private *' but argument is of type 'struct sja1105_private *'
static inline void sja1105_tas_teardown(struct sja1105_private *priv) { }
^~~~~~~~~~~~~~~~~~~~
drivers/net/dsa/sja1105/sja1105_main.c: In function 'sja1105_probe':
>> drivers/net/dsa/sja1105/sja1105_main.c:2215:20: error: passing argument 1 of 'sja1105_tas_setup' from incompatible pointer type [-Werror=incompatible-pointer-types]
sja1105_tas_setup(priv);
^~~~
In file included from drivers/net/dsa/sja1105/sja1105.h:23:0,
from drivers/net/dsa/sja1105/sja1105_main.c:24:
drivers/net/dsa/sja1105/sja1105_tas.h:38:20: note: expected 'struct sja1105_private *' but argument is of type 'struct sja1105_private *'
static inline void sja1105_tas_setup(struct sja1105_private *priv) { }
^~~~~~~~~~~~~~~~~cc1: some warnings being treated as errors
vim +/sja1105_tas_teardown +1731 drivers/net/dsa/sja1105/sja1105_main.c
1726
1727 static void sja1105_teardown(struct dsa_switch *ds)
1728 {
1729 struct sja1105_private *priv = ds->priv;
1730
> 1731 sja1105_tas_teardown(priv);
1732 cancel_work_sync(&priv->tagger_data.rxtstamp_work);
1733 skb_queue_purge(&priv->tagger_data.skb_rxtstamp_queue);
1734 sja1105_ptp_clock_unregister(priv);
1735 sja1105_static_config_free(&priv->static_config);
1736 }
1737
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Vladimir Oltean
maintainerRe: [PATCH v2 net-next 6/7] net: dsa: sja1105: Configure the Time-Aware Scheduler via tc-taprio offload
Quoted 14 lines · kbuild test robot
Hi Vladimir,
I love your patch! Yet something to improve:
[auto build test ERROR on net-next/master]
url:
https://github.com/0day-ci/linux/commits/Vladimir-Oltean/tc-taprio-offload-for-SJA1105-DSA/20190914-154650
config: i386-randconfig-b001-201936 (attached as .config)
compiler: gcc-7 (Debian 7.4.0-11) 7.4.0 reproduce:# save the attached .config to linux build tree make ARCH=i386
Quoted 10 lines
If you fix the issue, kindly add following tag
Reported-by kbuild test robot <[email protected]>All error/warnings (new ones prefixed by >>):
In file included from drivers/net/dsa/sja1105/sja1105.h:23:0, from drivers/net/dsa/sja1105/sja1105_spi.c:8:Quoted 3 lines
Quoted 3 lines
drivers/net/dsa/sja1105/sja1105_tas.h:38:45: warning: 'structsja1105_private' declared inside parameter list will not be visible outside of this definition or declaration
Oops, I didn't realize I need the "struct sja1105_private;" forward-declaration even if CONFIG_NET_DSA_SJA1105_TAS is not defined. Never mind, I'll just make all prototypes take "struct dsa_switch *ds" so the forward-declaration won't be needed at all. I'll send out a v3 soon.
Quoted 64 lines
static inline void sja1105_tas_setup(struct sja1105_private *priv) { } ^~~~~~~~~~~~~~~ drivers/net/dsa/sja1105/sja1105_tas.h:40:48: warning: 'struct sja1105_private' declared inside parameter list will not be visible outside of this definition or declaration static inline void sja1105_tas_teardown(struct sja1105_private *priv) { } ^~~~~~~~~~~~~~~ -- In file included from drivers/net/dsa/sja1105/sja1105.h:23:0, from drivers/net/dsa/sja1105/sja1105_main.c:24:Quoted 3 lines
Quoted 3 lines
drivers/net/dsa/sja1105/sja1105_tas.h:38:45: warning: 'structsja1105_private' declared inside parameter list will not be visible outside of this definition or declarationstatic inline void sja1105_tas_setup(struct sja1105_private *priv) { } ^~~~~~~~~~~~~~~ drivers/net/dsa/sja1105/sja1105_tas.h:40:48: warning: 'struct sja1105_private' declared inside parameter list will not be visible outside of this definition or declaration static inline void sja1105_tas_teardown(struct sja1105_private *priv) { } ^~~~~~~~~~~~~~~
drivers/net/dsa/sja1105/sja1105_main.c: In function 'sja1105_teardown':Quoted 3 lines
Quoted 3 lines
drivers/net/dsa/sja1105/sja1105_main.c:1731:23: error: passing argument 1 of 'sja1105_tas_teardown' from incompatible pointer type [-Werror=incompatible-pointer-types]sja1105_tas_teardown(priv); ^~~~ In file included from drivers/net/dsa/sja1105/sja1105.h:23:0, from drivers/net/dsa/sja1105/sja1105_main.c:24: drivers/net/dsa/sja1105/sja1105_tas.h:40:20: note: expected 'struct sja1105_private *' but argument is of type 'struct sja1105_private *' static inline void sja1105_tas_teardown(struct sja1105_private *priv) { } ^~~~~~~~~~~~~~~~~~~~
drivers/net/dsa/sja1105/sja1105_main.c: In function 'sja1105_probe':Quoted 3 lines
Quoted 3 lines
drivers/net/dsa/sja1105/sja1105_main.c:2215:20: error: passing argument 1 of 'sja1105_tas_setup' from incompatible pointer type [-Werror=incompatible-pointer-types]sja1105_tas_setup(priv); ^~~~ In file included from drivers/net/dsa/sja1105/sja1105.h:23:0, from drivers/net/dsa/sja1105/sja1105_main.c:24: drivers/net/dsa/sja1105/sja1105_tas.h:38:20: note: expected 'struct sja1105_private *' but argument is of type 'struct sja1105_private *' static inline void sja1105_tas_setup(struct sja1105_private *priv) { } ^~~~~~~~~~~~~~~~~cc1: some warnings being treated as errors
vim +/sja1105_tas_teardown +1731
drivers/net/dsa/sja1105/sja1105_main.c1726
1727 static voidsja1105_teardown(struct dsa_switch *ds)
1728 {
1729 structsja1105_private*priv =ds->priv;
1730Quoted 1 line
1731
sja1105_tas_teardown(priv);1732
cancel_work_sync(&priv->tagger_data.rxtstamp_work);
1733skb_queue_purge(&priv->tagger_data.skb_rxtstamp_queue);
1734sja1105_ptp_clock_unregister(priv);
1735sja1105_static_config_free(&priv->static_config);
1736 }
1737
Quoted 6 lines
---
0-DAY kernel test infrastructure Open Source Technology
Center
https://lists.01.org/pipermail/kbuild-all Intel
Corporation
Thanks,
-Vladimir
Vladimir Oltean
maintainerPATCH v2 7/7 docs: net: dsa: sja1105: Add info about the time-aware scheduler
In reply to Vladimir OlteanWhile not an exhaustive usage tutorial, this describes the details needed to build more complex scenarios.
---
Changes since v1:
- Patch is new.
Documentation/networking/dsa/sja1105.rst | 90 ++++++++++++++++++++++++ 1 file changed, 90 insertions(+)
Documentation/networking/dsa/sja1105.rst +90 -0
@@ -146,6 +146,96 @@ enslaves eth0 and eth1 (the DSA master of the switch ports). This is because in this mode, the switch ports beneath br0 are not capable of regular traffic, and are only used as a conduit for switchdev operations. +Offloads +======== + +Time-aware scheduling +--------------------- + +The switch supports a variation of the enhancements for scheduled traffic +specified in IEEE 802.1Q-2018 (formerly 802.1Qbv). This means it can be used to +ensure deterministic latency for priority traffic that is sent in-band with its +gate-open event in the network schedule. + +This capability can be managed through the tc-taprio offload ('flags 2'). The +difference compared to the software implementation of taprio is that the latter +would only be able to shape traffic originated from the CPU, but not +autonomously forwarded flows. + +The device has 8 traffic classes, and maps incoming frames to one of them based +on the VLAN PCP bits (if no VLAN is present, the port-based default is used). +As described in the previous sections, depending on the value of +``vlan_filtering``, the EtherType recognized by the switch as being VLAN can +either be the typical 0x8100 or a custom value used internally by the driver +for tagging. Therefore, the switch ignores the VLAN PCP if used in standalone +or bridge mode with ``vlan_filtering=0``, as it will not recognize the 0x8100 +EtherType. In these modes, injecting into a particular TX queue can only be +done by the DSA net devices, which populate the PCP field of the tagging header +on egress. Using ``vlan_filtering=1``, the behavior is the other way around: +offloaded flows can be steered to TX queues based on the VLAN PCP, but the DSA +net devices are no longer able to do that. To inject frames into a hardware TX +queue with VLAN awareness active, it is necessary to create a VLAN +sub-interface on the DSA master port, and send normal (0x8100) VLAN-tagged +towards the switch, with the VLAN PCP bits set appropriately. + +Management traffic (having DMAC 01-80-C2-xx-xx-xx or 01-19-1B-xx-xx-xx) is the +notable exception: the switch always treats it with a fixed priority and +disregards any VLAN PCP bits even if present. The traffic class for management +traffic is configurable through ``CONFIG_NET_DSA_SJA1105_HOSTPRIO``, which by +default has a value of 7 (highest priority). + +Below is an example of configuring a 500 us cyclic schedule on egress port +``swp5``. The traffic class gate for management traffic (7) is open for 100 us, +and the gates for all other traffic classes are open for 400 us:: + + #!/bin/bash + + set -e -u -o pipefail + + NSEC_PER_SEC="1000000000" + + gatemask() { + local tc_list="$1" + local mask=0 + + for tc in ${tc_list}; do + mask=$((${mask} | (1 << ${tc}))) + done + + printf "%02x" ${mask} + } + + if ! systemctl is-active --quiet ptp4l; then + echo "Please start the ptp4l service" + exit + fi + + now=$(phc_ctl /dev/ptp1 get | gawk '/clock time is/ { print $5; }') + # Phase-align the base time to the start of the next second. + sec=$(echo "${now}" | gawk -F. '{ print $1; }') + base_time="$(((${sec} + 1) * ${NSEC_PER_SEC}))" + + tc qdisc add dev swp5 parent root handle 100 taprio \ + num_tc 8 \ + map 0 1 2 3 5 6 7 \ + queues 1@0 1@1 1@2 1@3 1@4 1@5 1@6 1@7 \ + base-time ${base_time} \ + sched-entry S $(gatemask 7) 100000 \ + sched-entry S $(gatemask "0 1 2 3 4 5 6") 400000 \ + flags 2 + +It is possible to apply the tc-taprio offload on multiple egress ports. There +are hardware restrictions related to the fact that no gate event may trigger +simultaneously on two ports. The driver checks the consistency of the schedules +against this restriction and errors out when appropriate. Schedule analysis is +needed to avoid this, which is outside the scope of the document. + +At the moment, the time-aware scheduler can only be triggered based on a +standalone clock and not based on PTP time. This means the base-time argument +from tc-taprio is ignored and the schedule starts right away. It also means it +is more difficult to phase-align the scheduler with the other devices in the +network. + Device Tree bindings and board design =====================================
Signature · 1 line
2.17.1